Is there a need to consider the relativistic mass when calculating gravitational potential energy? The equation for gravitational potential energy is:
$$V(r) ={GMm\over r}$$
In the reference frame where $M$ is stationary. I am wondering for mass $m$ here do we need to consider its relativistic mass? And what would be the reason behind it? 
 A: General relativity does not describe gravity using a scalar potential as in Newtonian gravity. GR's closest equivalent to the Newtonian potential is actually the metric.
No, you don't get a description of the sources of the gravitational field in GR by considering relativistic mass rather than mass. The source of the gravitational field in GR is the stress-energy tensor, not a scalar mass or mass-energy density. The mass-energy density is one of the 16 components of the stress-energy tensor.
A: The answer is a big NO. I will show this with one (very naive) counter example.
Assume that we should use relativistic mass in $v=-\frac{GMm}{r}$. Let's say S is inertial observer who moves in $z$ direction with veloctiy $u$. Sun is at the rest at $(x,y,z)=(0,0,0)$ and Earth orbits the Sun in $xy$ plane with velocity $w$. Now, from S's perspective Earth and Sun move in $z$ direction with velocity $-u$ and based on our assumption we should use relativistic mass for Earth and Sun. The velocity of Earth in $xy$ plan is small enough compare to $-u$ so it's safe to neglet it for now. Suppose that Sun's invariant mass is $M$ and Earth's invariant mass is $m$. According to S, their relativistic mass will be $M'=\gamma_uM$ and $m'=\gamma_um$ respectively. If we put them in $v$ we will see that $v'=-\gamma_u^2\frac{GMm}{r}$ which is a lot greater than $v$. The problem is, Earth's velocity in $xy$ plane will become slower by a $\gamma_u$ factor from S's point of view. You can see it from velocity addition formula. Now, $F'=-\nabla_rv'$ So $F'_g=\gamma_u^2\frac{GMm}{r^2}$. 
Earth-Sun system is in equilibrium (that's centrifugal force is equal to gravitional force in Sun's frame hence Earth doesn't change its orbit, roughly), So in Sun's frame we have $F_g-F_c=\frac{GMm}{r^2}-\frac{mw^2}{r}=0$. According to first special relativity principle, Earth should not change its orbit in other inertial frames like S, that's $F'_g-F'_c=0$, but unfortunately it's not the case because $F'_g-F'_c=\gamma_u^2\frac{GMm}{r^2}-\frac{mw^2}{\gamma_ur}\ne 0$ clearly. Which shows that assuming relativstic mass is not a good idea.
P.S: Actually, instead of velocity addition formula, one can check Earth's motion in $xy$ plane from S's point of view with time dilation formula, That's, Earth orbits the Sun every $365$day from Sun's point of view, So from S's perspective it should take $\gamma_u 365$day for earth to complete its rotation. That would mean Earth's angular velocity is decreased by a gamma factor. If you want to consider relativistic effect, you will need Einstein field equations.
A: I would start with the metric line element for the nonrotating central gravitational field
$$
ds^2=(1-2m/r)dt^2-(1-2m/r)^{-1}dr^2-r^2(d\theta^2 + sin^2\theta d\phi^2),
$$
where $m=GM/c^2$. The variation of this is zero, where $ds$ is the proper time and is defined at an extremum. It is really a form of action. We can further see this if we divide by $ds^2$ with the four-velocities defined by $U_t=dt/ds$, $U_r = dr/ds$, $U_\theta = d\theta/ds$ and $U_\phi=d\phi/ds$. This gives
$$
1=(1-2m/r)U_t^2-(1-2m/r)^{-1}U_r^2-r^2(U_\theta^2 + sin^2\theta U_\phi^2),
$$
and clearly the variation of $1$ is zero. Now factor out $U_t=dt/ds$
$$
1=[(1-2m/r)-(1-2m/r)^{-1}v_r^2-r^2(v_\theta^2 + sin^2\theta v_\phi^2)]U_t^2,
$$
where we have the coordinate velocities $v_r = dr/dt$, $v_\theta = d\theta/dt$ and $v_\phi=d\phi/dt$.
Now let us for simplicity consider a nice circular orbit so that $v_r=0$ and we shall set this on a plane with $\theta=\pi/2$ and is constant. Now we have
$$
1=[(1-2m/r)-r^2\omega^2]U_t^2,
$$
where $v_\phi=\omega$ the angular velocity. We can see that obviously $U_t=[(1-2m/r)-r^2\omega^2]^{-1/2}$. This is a sort of gravitational Lorentz $\gamma$-factor. 
Now I will start writing this with $m=GM/c^2$ and us small $m$ for a test mass. I now multiply by this mass-energy $mc^2$
$$
mc^2=[(mc^2-2GMm/r)-mr^2\omega^2]U_t^2.
$$
Now do a little algebra
$$
\frac{1}{2}mc^2(U_t^2-1) = \left(\frac{1}{2}mr^2\omega^2+GMm/r\right) U_t^2.
$$
This is a sort of Lagrangian. For $U_t = 1$ we obtain a Newtonian type of result.
So this illustrates some of the connections between spacetime transformations and $-GMm/r$. In some ways the potential energy is really replaced with time dilation. The deeper a clock is in a gravity well the slower it appears. This is the main role of $-GMm/r$ that in a weak gravity limit is the Newtonian gravity potential.
