Negative potential energy of gravity

Does the negative potential energy in the gravitational field have to be considered in calculating the total mass of the system in question (because of $E=mc^2$)?

If so it seems to me that the adding this negative energy to the calculation would slightly diminish the strength of the gravitational force over long distances.

-

Depends on what you're doing. General relativity handles it for you, in the sense that the Einstein field equation links geometry to the non-gravitational stress-energy tensor. That general relativity is non-linear can be interpreted in part as gravity itself contributing to gravity, but it's generally not even possible to localize gravitational energy in a coordinate-independent manner. The equivalence principle forbids it, since freefall is locally inertial and so any gravitational field can be locally 'transformed away'.

That said, for weak fields the parametrized post-Newtonian formalism includes nonlinearity. For the static case, the first few terms are: $$\mathrm{d}s^2 = -\left(1+2\Phi + 2\beta\Phi^2\right)\mathrm{d}t^2 + \left(1-2\gamma\Phi\right)\left(\mathrm{d}x^2+\mathrm{d}y^2+\mathrm{d}z^2\right)\text{,}$$ where general relativity predicts $\beta = \gamma = 1$.

This is treated in a lot more detail in, e.g., the textbook of Misner, Thorne, and Wheeler. In their notation, the parameter describing the contribution of the gravitational potential itself is called $\beta_2$, cf. Box 39.2 and Section 39.8 to see how it works quantitatively. The wikipedia link provides an overview.

-