The Preference for Low Energy States The idea that systems will achieve the lowest energy state they can because they are more "stable" is clear enough. My question is, what causes this tendency? I've researched the question and been unable to find a clear answer, so I was hoping someone could explain what's going on behind the scenes here (or if nothing is and it's just an observed law that isn't explainable).
 A: Large systems with many degrees of freedom (e.g. a ball consisting of many molecules) tend to settle into low energy states. This is a direct consequence of two fundamental laws, the first and second laws of thermodynamics: energy conservation and entropy increase.
A system with many degrees of freedom can be in many different microscopic states (think about a ball for which each molecular position and vibration etc is specified). Each such feasible micro state is equally likely. However, what we typically observe is not a micro state, but a coarse-grained description (the position of the ball) corresponding to incredibly many micro state. Certain macro states correspond to far fewer micro states than other macro states. As nature has no preference for any of these micro states, the latter macro states are far more likely to occur. The evolution to ever more likely macro states (until the most likely macro state, the equilibrium state, is reached) is called the second law of thermodynamics.
The decrease of potential energy is the consequence of the first (energy conservation) and second (evolution to more likely macro states) law of thermodynamics. As macro states with a lot of energy stored in heat (our ball with random thermal motion of its molecules) contain many more micro states and are therefore much more likely, energy tends to get transferred from potential energy to thermal energy. This is observed as a tendency towards a decrease in potential energy.
A: The probability of finding a system in a state with energy $E$ is $P(E) = \exp(-\beta E)/Z$, where $\beta = (kT)^{-1}$, $k$ being the Boltzmann constant and $T$ being the absolute temperature. $Z$ in the formula for $P(E)$ is the canonical partition function. For our purpose, we can consider it as a factor introduced to ensure that $0 \le P(E) \le 1$. The formula for $P(E)$ tells you immediately that higher energy states are not favored.
But that's just an explanation from the formula. You can refer to a book on statistical mechanics, for example Kittel's Elementary Statistical Physics (sec. 11) to find out how it is derived. Alternatively, you can go through the article on "Boltzmann factor" on wikipedia.
A: I will address such sample systems as a point (or a small metal ball) rolling or bouncing on some hard surface with hills and pits, and an atom which can be either in excited or in basic state.
I. If we consider an ideal closed system, then the enegly is conserved. But real systems do not (exactly) behave this way. For a macroscopic mechanic motion we can add non-ideality, that is, friction or other kind of energy dissipation. But on a microscopic level there is no friction as far as we know. Atom falls down to its basic state due to radiation and collisions with other atoms.
II. If we try to generalize the cases of going to the lowest energy from real-life samples, we could come to the notion of degrees of freedom. A ball has mechanical degrees of freedom when it moves, and some energy assigned to them. An atom has its internal degree (or degrees) of freedom when it is excited. And that energy tends to leak to some other degrees of freedom, which are not accounted to the considered system. For a ball, that would be internal motions in the matter, which we call heat and sound and inelastic deformations. For an atom, that would be degrees of freedom of colliding atoms, and internal degrees of freedom of the electromagnetic field, which we call electromagnetic radiation (or waves or photons or light). I consider only two samples but the idea is very general: systems go to the lowest energy because they share their energy with something else, and that 'something' has so many degrees of freedom that energy is divided into indiscernible small portions. Sometimes the energy is not actually divided (an atom emits a single photon), but it escapes so fast that we lose sight of it, and never returns. Then it is the same: our system has fallen to the lowest energy state.
III. But let us consider a logical possibility: what if those degrees of freedom, which are external to our system, do posess some energy themselves? Would not they share it back with our system, raising it from the lowest energy to some higher energy? They would. This is called the finite temperature of environment, and thermal equilibrium, when the energy cannot fall down entirely because it is risen with the same rate as it falls. That is where the formula Amey Joshi quoted applies. The energy does not stay fixed on some level, as it would be for two balanced energy flows. Instead, it rises and falls with random steps at random moments, and performs a random walk similar to brownian motion. In this walk, there are values that are visited more frequently than the others, and they are states of the lower energy - but not the exactly lowest. This tendency can be pronounced more strongly or weakly, depending on the value of temperature. If the temperature is extremely low, everything behaves as it was described in the II section. And if the temperature is extremely high, systems do not tend to lower energies at all. How do we estimate those temperatures? We take the formula $\langle E\rangle=\tfrac{1}{2}k_B T$ for the average energy for the degree of freedom. For the metal ball of mass 0.1 gram, jumping unstoppingly to the average height of 1 meter, $\langle E\rangle=10^{-3}$ Joules, and the needed temperature is $\sim 10^{20}$ Kelvins (much higher than it is needed to evaporate the ball, make it to the plasma, then to free particles, and finally to the soup of unconfined quarks - which is about $\sim 10^{13}$ Kelvins). That is why we do not see such behavior in everyday life. For atoms, though, that happens more often - we need the temperature of only hundreds and thousands of degrees to make atoms permanently excited, and molecules are in endless motion at the room temperature.
A: The other answers tackle the statistical/thermodynamic aspect. I will tackle the "falling apple " aspect.
Why does the apple fall?
From this observation onwards nature was modeled mathematically as interactions between masses, in this case, charges in the electromagnetic case etc.
The observations of gravitational  interactions led to a mathematical model that incorporated as laws the observation of conservation of energy and momentum.
The solutions of these equations necessitated the concept of potential energy. The statement " energy is conserved"  is true in the the sum of energies, kinetic and potential, in the problem.
In the case of the apple, it has potential energy before it falls which turns into kinetic energy with the fall, the sum being constant. It reaches the ground as the lowest energy state because gravitational interaction is attractive. It cannot go through thr ground because of the opposing electromagnetic forces of the ground molecules, which are much stronger then the gravitational force still pulling the apple to the center of the earth.
For repulsive forces the opposite holds. The potential energy of two electrons decreases as they repulse each other turning potential energy to kinetic energy.
In either case a particle can stay in a high potential energy state only if there are other forces holding it there: in the case of the apple the cohesive forces of the stem before gravity overcomes them. Particles will end up in the lowest potential energy state available.
Conservation of energy ( and other conservation laws) are based on experimental observations. The theories model well these observations when the laws are assumed as unbreakable , as they are in the mathematical formulations. Thus in a sense your 

it's just an observed law that isn't explainable

is the true answer in this fundamental example. Once though one has the individual particle interactions, their statistical behavior is as explained in the other answers.
A: You can allso call it a force free system, meaning that the net force in the system is 0.
Lets asume you have simple quasi-closed homogen system filled with helium gas. Then we asume you introduce a force on small volume of the gas in the system.
This increases the energy with in this volume.
When the excited particles, that we asume have the same energy, bounce in to each other each particle keeps the same energy.
When a excited particle bounce in to a unexcited particle the low energy particle increase its energy while the high energy particle will decrease its energy.
This means that there is a higher probability for the energy to become evenly distributed than to be concentrated in a small part of the system. This phenomenon of a subsystem moving towards a lower energy state is called a instability.
This is why you have the second law of thermodynamics by the way.
First we asumed very simple laws. If we change the laws from simple to very complex the force free state might look very much diffrent.
If we asume we have a ressistive plasma (like a not fully ionized gas) instead of helium the we have to count with electric forces, magnetic forces and the very complex nature of the plasma. Then the system might apear to have energy pockets here and there, but that is just the complex systems way of reducing the net force to 0.
You should expect very complex instabilites and currents everywhere due to some instabilies. Out of the currents you will get a mess of more complex instabilies.
