Potential energy has absolutely nothing to do with stress-energy or pressure. The following reference is a good source about the origin of the pressure term in the stress-energy tensor: "Momentum due to pressure: A simple model" by Kannan Jagannathan in American Journal of Physics 77, 432 (2009); http://dx.doi.org/10.1119/1.3081105
Potential energy itself (and all direct action at a distance) is generally incompatible with relativity. If something gains momentum then it generally has to get the momentum from something else at the exact same time and place.
So what happens with forces such as electromagnetism is that when a positively charged particle moves in the opposite direction of the electric field it loses momentum and energy and the fields gain it. Later the energy and momentum move with the fields until it meets another charged particle, if that charged particle is positively charged and going in the direction of the electric field the particle gains energy and momentum and the fields lose it. There is no such thing as potential energy. Energy and momentum are simply transferred between different things.
Similarly for contact forces, you can exchange energy and momentum directly between objects. All of the above holds in special relativity, so also holds in very small regions of spacetime.
Now gravity is different. It's really about how different regions piece together. Firstly, different regions can be curved and hence piece together local regions in a particular way. This can happen even when the stress-energy tensor is zero.
What the stress-energy tensor does is allow spacetime to curve differently than it otherwise would.
And again it's not potential energy. It's just a curved spacetime curving the natural way or curving a different way because of the presence of stress-energy.