What is potential energy truly? I have a problematic question for which I have been unable to attain a satisfactory answer. What is potential energy truly? 
-
I have read about how potential energy can be seen as the "highering" of an object within a field, be it gravitational, electromagnetic, etc. 
In a way, an object is lifted to a higher position, and when it eventually falls down again energy is released back.
The object does not even have to be "lifted" beforehand, as with the case of asteroids. 
However, since energy cannot be destroyed, it must somehow be stored, yet how does energy truly store itself within a field? It seems hard for me to grasp that energy can physically be stored within the positioning of an object?
I can't really get my head around how a field conserves energy within it. Kinetic energy on the other hand, can be, for me, intuitively understood, as the object really is "moving" and energy manifests itself through motion.
Can anyone elaborate on this, perhaps give reasoning as to why a field can in fact "store" energy? Please point out any fallacious or naive views I seem to hold.
 A: 
It seems hard for me to grasp that energy can physically be stored within the positioning of an object?

If you place a huge boulder at the edge of a large cliff, and give it a small poke just as a bear is ambling below, the bear below will be completely obliterated upon impact by the boulder. There was definitely energy released in the concomitant bear annihilation; where did it come from? 
Well, the energy was physically stored in the positioning of the boulder.
Unfortunately, if you are looking for a deeper physical intuitive understanding of why this is, you probably won't find one. You can devise various formalisms, such as the notion of an underlying gravitational potential well $U_\text{grav}(h)=mgh$, but to some extent, you simply might have to accept as an axiom that energy can be stored in the form of work performed against a potential, since that seems to be how physical reality operates.
There are probably more mathematically complicated or physically elegant ways to encode this basic idea, so I'll await other users explanations, but hopefully the above gives some small measure of insight.
A: Given the law of conservation of energy, it is often evocative and helpful to imagine energy as a kind of permanent "stuff" and along with this an imagination of "where" it is.
But this doesn't always work, so another "answer", complementary to the others, is the following. 
What is it "truly"?  It is simply a manifestation of a system's time shift invariance. It is a manifestation of a system's symmetry.
A modern insight into the "reason" for conservation of energy in a system is Noether's theorem. If a system's Lagrangian has a continuous symmetry, then, by Noether's theorem, there is one conserved quantity for each such symmetry. Most physical systems "don't care" where we put our $t=0$ origin for our co-ordinate system. The $t=0$ origin's exact location is an artifact of our description of a physical system, not of the system itself. Therefore, for most systems, if we impart the time shift $t\to t+\alpha$ for any $\alpha\in\mathbb{R}$, then our equations describing the system's physics do not change. 
The conserved quantity that arises from a system's time shift invariance symmetry by Noether's theorem is what physicists call energy.
So when some moving objects in a system slow down and you want to know "where" their kinetic energy has gone to, you can instead say that properties of those and other objects have changed so as to uphold the total system's invariance to time origin shift. Or, there must be a corresponding "balancing" change by dent of a certain symmetry. The question of "where" then doesn't arise.
This is a fairly abstract thinking, but you you might find it helpful in some situations - I urge you to think of it as complementing rather than replacing more elementary intuitive notions of energy as a "stuff".

As user Christoph says (thanks Christoph):

note that we need to answer the 'where' question for anything besides gravitational energy if we want to apply general relativity...

This is very true and it also serves to illustrate the limitations of the Noether symmetry idea, at least as far as energy is concerned. What Christoph means is that in General Relativity, one must indeed "locate" energy by writing down the so called stress energy tensor $T$ to be the source term in the Einstein field equation. The $0 0$ component of $T$ is the energy density in space per unit volume as a function of position and time (i.e. a manifest assertion of "where the energy is").
In General Relativity, however, my answer does not apply, because there is in general no time shift invariance symmetry anymore. Global energy need not be conserved in general relativitstic systems - although there is still a local conservation of energy, meaning roughly that energy conservation approximately holds for systems that are small enough in both their spatial and temporal extent.
A: Let's start about what is really a field (gravitational, electromagnetic,...). It is actually a function of space and time. Every point of the field has a special property. In the  electromagnetic case  for example, there is a function we call the electromagnetic field and it describes the electromagnetic force that the field will cause at any external charged particle that exist at a specific point of the field. As you can understand, since in principle the field doesn't have to be homogeneous, the different points of the field are not equivalent. If I bring the same charged particle at different points, different force will be applied to them. Another way to describe the same space is the potential. Instead of the force (which is a vector), the potential describes again every space-time point with a number, that number being the energy one needs to bring the particle at a specific point of the field from outside the field (Outside the field could mean infinity or just a point where we put the potential  to be zero). The two descriptions are equivalent. Since now the particle can move inside the field, it passes from points of different potential. Energy is conserved since, in order to be able to move, it has kinetic energy, that either it obtains when moving from a point of higher potential energy to a point of lower potential energy, either somebody else has to give it to move the particle towards a point of a higher potential. This picture is consistent and it is a high school exercise to calculate that at every point the sum of potential energy and kinetic energy is conserved (except if there is some external energy offered or subtracted that you have to take into account).
P.S. Try also to understand the difference of potential and potential energy.
A: Energy is not a substance. It is just a number one can calculate. One does not need to imagine energy as being stored somewhere, at least classically.
That being said, imagining energy as a substance can help in certain situations, but it is not often obvious "where" the energy is. One might say energy is stored in the field, but I feel this isn't completely satisfactory.
I suspect most practicing physicists don't concern themselves with this question much because it doesn't matter; you know how to calculate it, and you know how to use it. Done.
A: Potential energy is energy due to a system's configuration, a very abstract concept that doesn't necessarily translate directly to physical reality. However, gravity aside, we do believe that in classical theories we should in principle be able to tell where and how the different contributions to total energy that we conveniently describe as potential energy are stored.
The most instructive example is probably electromagnetism: If you forcibly re-arrange a set of charges, you perform work against the electromagnetic field. That energy gets stored within the field and can be released later on. As the electromagnetic field has a well-defined energy density, you can actually tell where it is (cf Poynting's theorem).
Gravity is more problematic because in general relativity, gravitational energy is delocalized and cannot be associated with a density. Arguably, the basic idea is still the same. In fact, there are (mostly) equivalent reformulations of GR that allow us to define such an energy density (bimetric theories, teleparallel gravity). However, for energy conservation to hold, even in these theories there's still a contribution to total energy that can't be localized. I tend to think of this contribution as the generalization of the centrifugal potential.
A: WARNING: Thought experiment ahead!


*

*Have Scottie beam your rock halfway between milkyway and andromeda. Does your rock have potential energy? The practical (not absolute) answer is NO ... but ... it turns into a big YES when some object starts exerting a gravity pull on the rock.


Potential energy only has a meaning in a "more than one body" system so it's really not a 'position' thing but a 'relative position' thing.
Just as 'glance' stated, it's merely book-keeping ;-)
A: Why not think about work and energy? Presumably you have a good hold on forces, since they are very intuitive. If you push an object over a long distance, you are doing work on that object. This work is energy.
In more detail, the work done on an object if you push a distance $\Delta x$ with a force $F$ is
$$W=F \Delta x$$
(ignoring complications due to directions, etc. I am pushing in the same direction the object is moving). 
Well, for a conservative force, the definition of the potential energy associated with that force is
$$-\frac{\Delta U}{\Delta x}=F$$
So....
$$F\Delta x=W=-\Delta U=F\Delta x$$
So, you push on an object with a force, you are doing work, and adding energy. How is the energy "stored"? In the forces! 
A: Gravity contracts our unit of length and dilates our unit of time. The length contraction reduces the total energy of mass, while the time dilation increases its total momentum.
When a mass falls freely through a gravitational field, there is no net force acting on it, so it's total potential to do work remains constant. But if we describe that potential in units of force × distance (E) we find it is decreasing. On the other hand, if we describe it in units of force × time ($\rho$), we find it is increasing.
The decrease in total energy due to length contraction is what we call gravitational potential energy.
That seems a bit hard to believe at first, because we know that gravitational length contraction, like its counterpart, gravitational time dilation, is extremely small. However, while it has a factor of $c^2$ in its denominator, the total energy of mass has a factor of $c^2$ in its numerator, and they cancel out. We end up, over small height changes, with
$\Delta{E}=mc^2×g\Delta{h}/c^2$
Or more familiarly
$GPE=mgh$
To sum up, gravity, by changing the relationship between length and time, causes a change in the balance of total energy to total momentum of a mass. It is the swing in total energy that we refer to as GPE.
But I have to say that although all this talk of unit changes makes certain things clearer, it has become obvious to me that it is really caused by a gradient in light speed through space. Of course if we measure light speed in local units it will always look the same.
