Is a *difference* of potential energy relative to a frame of reference? If we consider an electrical field, or a gravitational field, and two points in this field, is the difference of potential between this two points depending of a frame of reference ?
It seems to me that in a uniform gravitational field ($\cong$ at the surface of Earth), the difference of potential between two points (fixed with respect to each other) is given by $mgh$, and does NOT depend on whether I'm moving or not relatively to the two points.
Now I tried to do a computation (I spare you the details, otherwise the question will be waaaay too long, but I have them if necessary), and I hit a problem: a difference of kinetic energy is frame-dependent, so a difference of potential energy should also be frame dependent.
But then, what is wrong in using (for instance) $mgh$? Because nowhere in there is my relative speed to the frame containing the two points. 
 A: In ordinary Newtonian physics, when one changes reference frame, the gravitational potential energy does not change. On the other hand, the total energy of a particle is not the same in two different frames of reference. The kinetic energy does change, when one goes from one frame to another, so the total energy changes by the same amount that the kinetic energy does. 
As a check, you can calculate the gravitational potential energy from scratch in each reference frame. This can be obtained from the integral of the force over the distance. It is because the gravitational force does not depend on velocity (in Newtonian physics) that one gets the same answer irrespective of the velocities involved. But see my added note below which qualifies this a bit.
There is a further good take-home message here. Notice that the energy of a particle should not be thought of as a property only to do with the particle, because it is relative. Any statement along the lines of "the particle has energy 1 joule" should be understood as a shorthand for "when observed in such-and-such a frame of reference, the particle has energy 1 joule".
Further remark
I hit "enter" on the above answer a little too quickly. There is a an important further point. The concept of potential energy only really works as an aid to calcualtion (in a simple way at least), when the potential at a given place is not itself a function of time in the reference frame under consideration. We can still imagine a potential well with the Earth at the middle, for example, but a moving potential well produces time-dependent forces and that means you don't expect that a particle going away and coming back to the same place will have the same change in its kinetic energy. In this sense the force is then "non-conservative". This terminology doesn't mean that energy conservation no longer applies, it does mean that the concept of potential energy no longer offers a quick and easy way to keep track of the energy movements. 
Here is another example. There is elastic potential energy in the strings of a tennis racket when it hits a tennis ball. But the change in momentum of the ball, when the racket hits the ball, depends not only on the extension of the strings (which leads to a given force) but also on the total time during which the force is applied. So here again, the concept of potential energy is not enough on its own to tell you how the overall energy conservation works out.
If you want the technical details for this area of physics, then look up "Lagrangian mechanics".
A: I may have misunderstood your question to be about relativity. I am leaving the original answer up below this one:
The gravitational potential energy does not depend on reference frame in classical physics, as you note it only depends on h which does not change between reference frames. You can not draw the conclusion that because kinetic energy is frame dependent potential energy must be too. Energy is conserved (the same at all times) but it is not invariant (the same in all reference frames).
Original answer:
If you have a mass at a distance $h$ from the ground in it's frame it has potential energy $mgh$ (if close to the ground). If we now shift into the reference frame of a rocket moving at velocity $v$, say, towards it along the line that goes through the center of the earth and the object the height that the object is above the ground will be length contracted to $h'=\sqrt{1-\frac{v^2}{c^2}}h$, and the object will have potential energy $mgh'$.
It is also important to point out that while energy is conserved (the same at different times) it is not invariant (the same in different reference frames). Rather, the energy of an object makes up the time component of a four dimensional vector called the four momentum, $p_\mu$, where the space components are the components of the normal momentum $\vec{p}$ (times the speed of light to ge the right units). When you change to a reference frame with a different velocity you "rotate" time and space into each other, and the components of $p_\mu$ change. 
However, they change in such a way that the length of $p_\mu$ remains the same. The length of a four vector is computed as the difference between the time and space components: $L^2=E^2-(\vec{p}c)^2$. According to Einstein this length is the mass energy of the object, $mc^2$, which gives us his famous relation $E^2=(pc)^2+(mc^2)^2$.
