What's the correct relativistic lagrangian when calculating the Hamiltonian? Introduction:
The action of a test particle in curved spacetime is
\begin{equation}
S=-m\int d\tau
\end{equation}
because we can't do a variation on proper time without changing the boundary terms, one switches to an arbitrary parameter $\lambda$
\begin{equation}
S=-m\int \sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}d\lambda
\end{equation}
The lagrangian is
\begin{equation}
L_0=-m\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}
\end{equation}
This lagrangian gives the right equations of motion
\begin{equation}
\frac{D}{d\tau} \frac{dx^\mu}{d\tau}=0
\end{equation}
where $\frac{D}{d\tau}=\frac{dx^\mu}{d\tau}\nabla_\mu$ is the total time derivative along the world line.
If one tries to calculate the Hamiltonian for this system one gets zero
\begin{equation}
H= \frac{\partial L_0}{\partial \dot{x}^\mu} \dot{x}^\mu-L_0 =0
\end{equation}
This can be traced back to the fact that the action is reparametrization invariant so the system is invariant under local time translations. The solution here is to replace the lagrangian with this alternative one
\begin{equation}
L=\frac{m}{2}g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}
\end{equation}
This Lagrangian gives the equations of motion
\begin{equation}
\frac{D}{d\lambda}\frac{d\dot{x}^\mu}{d\lambda}=0
\end{equation}
which implies that $\lambda$ is actually proper time. So it gives the same equations of motion as the previous lagrangian. Furthermore, if we calculate the Hamiltonian we now get
\begin{equation}
H=\frac{g_{\mu\nu}}{2m}p^\mu p^\nu
\end{equation}
where $p^\mu=m \frac{dx^\mu}{d\lambda}$.
Question:
Now let's say we have an action
\begin{equation}
S=-m_0\int d\tau  - \int d\tau V\big[x(\tau)\big]
\end{equation}
for some potential $V(x)$. The Lagrangian would be
\begin{equation}
L=-m_0\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}-\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}} V(x)
\end{equation}
The question is: If we want to replace this Lagrangian with another Lagrangian that gives the same equations of motion but also allows us to calculate a Hamiltonian (like the example in the introduction), how should we change this Lagrangian?
My Attempt:
This Lagrangian
\begin{equation}
L=-m_0\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}-\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}} V(x)
\end{equation}
gives the following equations of motion
\begin{equation}
\frac{D}{d\tau}\big[m(\tau) \frac{dx^\mu}{d\tau}\big]=-\nabla_\mu V(x)
\end{equation}
where $m(\tau)=m_0+V\big[x(\tau)\big]$. My attempt was to use this Lagrangian
\begin{equation}
L=\frac{m_0}{2}g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}-\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}} V(x). 
\end{equation}
Which gives this equations of motion
\begin{equation}
\begin{equation}
\frac{D}{d\tau}\big[m'(\tau) \frac{dx^\mu}{d\tau}\big]=-\nabla_\mu V(x)
\end{equation}
which are escencially the same with the exception that the new mass is
\begin{equation}
m'(\tau)=m_0\sqrt{g_{\mu\nu}\frac{dx^\mu}{d\lambda}\frac{dx^\nu}{d\lambda}}+V\big[x(\tau)\big]
\end{equation}
The mass term ends up with an extra factor of $\frac{d\tau}{d\lambda}$. I am not sure if I can now specialize to $\lambda=\tau$ to get the same equations of motion. Is that allowed? In the free particle case $\lambda$ turned out to be proper time due to the equations of motion, it wasn't something we imposed.
 A: I did not check whether your calculation of the equations of motion is correct, but there is an alternative way of looking at the problem which requires fewer calculations, but might give some insight. Your Lagrangian can be rewritten as
$L = -(m_0 + V(x))\sqrt{g_{\mu\nu}\frac{dx^\nu}{dt}\frac{dx^\mu}{dt}}$
Assuming V(x) is everywhere greater than $-m_0$ this can be rewritten as
$L = -m_0\sqrt{\tilde g_{\mu\nu}\frac{dx^\nu}{dt}\frac{dx^\mu}{dt}}$
where $\tilde g_{\mu\nu} = g_{\mu\nu}(1+V(x)/m_0)^2$
So you are simply calculating geodisics, but for a different metric. In case the equation $V(x) > - m_0$ does not hold, the action does not decrease monotonically along each path (since the action can then also take positive values), so you will get equations of motion, but there will not be any sensible interpretation of the action as a proper time.
A: *

*Assuming $m+V(x)>0$ then OP's Lagrangian
$$L_0~=~-(m+V(x))\sqrt{-g_{\mu\nu}(x)\dot{x}^{\mu}\dot{x}^{\nu}}\tag{A} $$
is of the standard form
$$L_0~=~-m\sqrt{-g_{\mu\nu}(x)\dot{x}^{\mu}\dot{x}^{\nu}}\tag{B} $$ by
changing the metric $g_{\mu\nu}(x)$ appropriately, as explained in Anton Quelle's answer.


*The Hamiltonian formulation of the Lagrangian (B) is explained in e.g. this, this, this, this & this related Phys.SE posts.
