Coupled Harmonic Oscillator - Solve by diagonalization I have a problem where I have two massive Particles $M$ and one particle with mass $m<M$. The two particles on the outside are coupled with the one in the middle with two springs.
The Hamiltonian for the system is given by:
\begin{equation}
H=\frac{p_1^2}{2M}+\frac{p_2^2}{2M}+\frac{p_3^2}{2m} + \frac{1}{2}k(x_3-x_1-d)^2+\frac{1}{2}k(x_2-x_3-d)^2
\end{equation}
($d$ is the length of the spring at equilibrium.)
I have replaced
\begin{equation}
q_1 = x_1\\
q_2 = x_2-2d\\
q_3 = x_3 -d
\end{equation}
and rewrote the potential as a Matrix equation:
\begin{equation}
\frac{k}{2}\vec{q}^T A\vec{q}
\end{equation}
with
\begin{equation}
A = \begin{pmatrix}
1 & 0 &-1\\
0 & 1 & -1\\
-1 & -1& 1\\
\end{pmatrix}\qquad \vec{q} = \begin{pmatrix}q_1\\q_2\\q_3\end{pmatrix}
\end{equation}
I can find  diagonal representation:
\begin{equation}
A = \begin{pmatrix}
1 & 0 &0\\
0 & 1+\sqrt{2} & 0\\
0 & 0& 1-\sqrt(2)\\
\end{pmatrix}
\end{equation}
With the corresponding eigenvectors
\begin{equation}
\begin{pmatrix}
-1\\1\\0
\end{pmatrix}\qquad
\begin{pmatrix}
-\frac{1}{\sqrt{2}}\\-\frac{1}{\sqrt{2}}\\1
\end{pmatrix}\qquad
\begin{pmatrix}
\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}\\1
\end{pmatrix}
\end{equation}
This probably means to switch to new coordinates
\begin{equation}
y_1 = -q_1+q_2\\
y_2 = -\frac{1}{\sqrt{2}}q_1-\frac{1}{\sqrt{2}}q_2+q_3\\
y_3 = \frac{1}{\sqrt{2}}q_1+\frac{1}{\sqrt{2}}q_2+q_3
\end{equation}
But what do I do with the momentum operators. They should change accordingly but I am confused as to how exactly
 A: I) As OP has noted, it is not enough to forget about the momentum variables and just diagonalize the position variables alone. Rather one should only use symplectic transformations.
In fact, if there is given a semipositive definite quadratic real Hamiltonian, one may show that there exists a real symplectic transformation that brings the Hamiltonian on diagonal form.
II) Further hints: 


*

*With OP's Hamiltonian 
$$H~=~T+V, \qquad T~=~\frac{p_1^2+p_2^2}{2M} + \frac{p_3^2}{2m}, \qquad  V~=~\frac{k}{2}(q^1-q^3)^2 +  \frac{k}{2}(q^2-q^3)^2, \tag{1}$$ 
choose position transformation of the form 
$$Q^+~=~ \frac{q^1+q^2}{2}-q^3,\qquad Q^-~=~ \frac{q^1-q^2}{2},\qquad Q^3~=~a_1q^1+a_2q^2+q^3,\tag{2} $$
where $a_1$ and $a_2$ are constants to be determined later.

*Extend the position transformation (2) to a canonical transformation 
$$ Q^i~=~\frac{\partial F_2(q,P)}{\partial P_i}, \qquad p_i~=~\frac{\partial F_2(q,P)}{\partial q^i},\tag{3}$$
in phase space for a suitable generating function $F_2(q,P)$ of type 2.

*Fix the constants $a_1$ and $a_2$ so that the kinetic energy $T$ becomes diagonal in the new momentum variables $P_i$.
References:


*

*V.I. Arnold, Mathematical methods of Classical Mechanics, 2nd eds., 1989; Appendix 6.

A: You solve the last 3 equations for the $x_i$ and express them as functions of the normal coordinates $y_i$. Then you differentiate these equations and multiply the velocities with the masses $m_i$ to obtain the momenta $$p_i=m_i\frac{dx_i}{dt}=m_i f_i(dy_1/dt,dy_2/dt,dy_3/dt)$$ These expression for the momenta in terms of the new coordinate velocities you insert into the original Hamiltonian.
