Hamiltonian of two coupled oscillators Lets say I have this system:

Two different masses with three different springs.
It's not very nice to do, but I can find the eigenvalues of this system (It's not nice because the two masses are different and the three springs are different).
The Hamiltonian is:
$$
H=\frac{p_{1}^{2}}{2m_{1}}+\frac{p_{2}^{2}}{2m_{2}}+\frac{k_{1}x_{1}^{2}}{2}+\frac{k_{2}x_{2}^{2}}{2}+\frac{k\left(x_{2}-x_{1}\right)^{2}}{2}
$$
and the expression for $\omega$ (normal mode's frequencies after diagnolization):
$$
\omega_{1,2}^{2}=\frac{-\frac{m_{2}k_{1}+m_{2}k+km_{1}+m_{1}k_{2}}{m_{1}m_{2}}\pm\sqrt{\left(\frac{m_{2}k_{1}+m_{2}k+km_{1}+m_{1}k_{2}}{m_{1}m_{2}}\right)^{2}-4\left(\frac{k_{1}k+k_{1}k_{2}+kk_{2}}{m_{1}m_{2}}\right)}}{2}
$$
Now, let's say I want to write the Hamiltonian in the terms of
$$
\omega_{1} , \omega_{2}
$$
How do I do that?
I don't see a way fixing the omegas to substitute into the Hamiltonian.
Thanks!
Edit for ytlu:
Hey :)
Thanks again, but still I don't understand. I don't mind finding the exact omegas. I just want to write the Hamiltonian in a form like that:
$$
H=\frac{\hat{p}_{1}^{2}}{2m_{1}}+\frac{\hat{p}_{2}^{2}}{2m_{2}}+\frac{m_{1}\omega_{1}^{2}\hat{x}_{1}^{2}}{2}+\frac{m_{2}\omega_{2}^{2}\hat{x}_{2}^{2}}{2}
$$
meaning, a decoupled Hamiltonian. But I'm not sure if it is right to write it and how to show it.
for example, If you have the same system but with all the masses equal and all the springs equal then you get
$$
\omega_{1}=\sqrt{3k/m}
$$
$$
\omega_{2}=\sqrt{k/m}
$$
and then it is straight forward to substitute the k into the Hamiltonian to get this form:
$$
H=\frac{\hat{p}_{1}^{2}}{2m_{1}}+\frac{\hat{p}_{2}^{2}}{2m_{2}}+\frac{m_{1}\omega_{1}^{2}\hat{x}_{1}^{2}}{2}+\frac{m_{2}\omega_{2}^{2}\hat{x}_{2}^{2}}{2}
$$
I'm looking for a way doing it when all the masses and springs are different
 A: Starting from the dynamical matrix for the normal mode of the coupled oscillation. The normal mode is defined as:
$$
   \begin{pmatrix} x_1(t) \\ x_2(t) \end{pmatrix} = e^{i\omega t}
    \begin{pmatrix} \xi_1 \\ \xi_2 \end{pmatrix}. \tag{1}
$$
And the equation for the normal mode:
$$
   -\omega^2 \begin{pmatrix} \xi_1 \\ \xi_2 \end{pmatrix} =
  \begin{pmatrix} -\frac{k_1 + k}{m_1} & \frac{k}{m_1} \\
         \frac{k}{m_2} & -\frac{k_2 + k}{m_2} \end{pmatrix}
    \begin{pmatrix} \xi_1 \\ \xi_2 \end{pmatrix} \equiv \mathbf A \vec \xi. \tag{2}
$$
The dynamical matrix $\mathbf A$ is not symmetrical. To fix this problem, we re-write Eq.1 as
$$
  \vec y \equiv \begin{pmatrix} \sqrt{m_1} x_1(t) \\ \sqrt{m_2} x_2(t) \end{pmatrix} = e^{i\omega t}
    \begin{pmatrix} \sqrt{m_1} \xi_1 \\ \sqrt{m_2}\xi_2 \end{pmatrix}.
$$
Equation 2 becomes:
\begin{align*}
   -\omega^2 \begin{pmatrix} \sqrt{m_1} \xi_1 \\\sqrt{m_2} \xi_2 \end{pmatrix} &=
  \begin{pmatrix} \sqrt{m_1} & 0 \\
         0 & \sqrt{m_2} \end{pmatrix}
    \begin{pmatrix} -\frac{k_1 + k}{m_1} & \frac{k}{m_1} \\
         \frac{k}{m_2} & -\frac{k_2 + k}{m_2} \end{pmatrix}
  \begin{pmatrix} \frac{1}{\sqrt{m_1}} & 0 \\
         0 & \frac{1}{\sqrt{m_2}} \end{pmatrix}
  \begin{pmatrix} \sqrt{m_1} & 0 \\
         0 & \sqrt{m_2} \end{pmatrix}
    \begin{pmatrix} \xi \\ \xi \end{pmatrix} \\
&=  \begin{pmatrix} -\frac{k_1 + k}{m_1} & \frac{k}{\sqrt{m_1 m_2}} \\
         \frac{k}{\sqrt{m_2 m_1}} & -\frac{k_2 + k}{m_2} \end{pmatrix}
   \begin{pmatrix} \sqrt{m_1} \xi_1 \\\sqrt{m_2} \xi_2 \end{pmatrix}  \\
\end{align*}
In term of vector $\vec y$, the dynamical matrix becomes symmetricl:
$$
-\omega^2 \vec y = \begin{pmatrix} -\frac{k_1 + k}{m_1} & \frac{k}{\sqrt{m_1 m_2}} \\
         \frac{k}{\sqrt{m_2 m_1}} & -\frac{k_2 + k}{m_2} \end{pmatrix}
   \begin{pmatrix} \sqrt{m_1} \xi_1 \\\sqrt{m_2} \xi_2 \end{pmatrix}  \equiv\mathbf B \vec y. \tag{3}
$$
Thus, we can diagonal the hamiltonian using the eigen vectors of matrix $\mathbf B$. They are mutually orthogonal.

Diagonalization of Hamiltonian: Write Hamiltoniam in terms of $\vec y$, defined as
$$
  \vec y(t) \equiv \begin{pmatrix} \sqrt{m_1} x_1(t) \\ \sqrt{m_2} x_2(t) \end{pmatrix}.
$$
The hamiltonian becomes
\begin{align*}
H &= \frac{1}{2} \left(\frac{\sqrt{m_1} dx_1}{dt}\right)^2 + \frac{1}{2} \left(\frac{\sqrt{m_2} dx_2}{dt}\right)^2 \\
&+ \frac{1}{2} \begin{pmatrix} \sqrt{m_1}x_1(t) & \sqrt{m_2}x_2(t) \end{pmatrix} \begin{pmatrix} +\frac{k_1 + k}{m_1} & -\frac{k}{\sqrt{m_1 m_2}} \\
         -\frac{k}{\sqrt{m_2 m_1}} & +\frac{k_2 + k}{m_2} \end{pmatrix}
    \begin{pmatrix} \sqrt{m_1} x_1(t) \\ \sqrt{m_2}x_2(t) \end{pmatrix}\\
 &= \frac{1}{2} \dot{\vec y}^T \dot{\vec y} + \frac{1}{2} \vec y^T \mathbf B \vec y
\end{align*}
Because $\mathbf B$ is a symmetrical matrix, its eigen vectors form a orthogonal matrix
$$
\mathbf R = \begin{pmatrix} \cos\theta & -\sin\theta \\
         \sin\theta & \cos\theta \end{pmatrix}
=\begin{pmatrix}\hat v_1^T \\
         \hat v_2^T \end{pmatrix}
$$
where $\hat v_1$ and $\hat v_2$ is two eigen vectors of $\mathbf B$:
\begin{align*}
   \mathbf B \hat v_1 &= \lambda_1 \hat v_1 = \omega_1^2 \hat v_1 \\
   \mathbf B \hat v_2 &= \lambda_2 \hat v_2 = \omega_2^2 \hat v_2
\end{align*}
$\mathbf R \vec y = \vec \eta$,  and $\vec y^T \mathbf R^T = \vec \eta^T$, with $\mathbf R^\dagger \mathbf R =\mathbf I$.
Apply the the transformation to the above Hamiltonian:
\begin{align*}
H &= \frac{1}{2} \dot{\vec y}^T \dot{\vec y} + \frac{1}{2} \vec y^T \mathbf B \vec y\\
&= \frac{1}{2} \dot{\vec y}^T \left( \mathbf R^T \mathbf R\right) \dot{\vec y} + \frac{1}{2} \vec y^T\left( \mathbf R^T \mathbf R\right) \mathbf B \left( \mathbf R^T \mathbf R\right)\vec y\\
&=  \frac{1}{2} \dot{\vec \eta}^T  \dot{\vec \eta} + \frac{1}{2} \vec \eta^T \mathbf R \mathbf B  \mathbf R^T \vec \eta\\
&=  \frac{1}{2} \left(\dot{\eta_1}^2 + \dot{\eta_2}^2\right) + \frac{1}{2} \left(\omega_1^2 \eta_1^2 + \omega_2^2 \eta_2^2 \right)
\end{align*}
where $\mathbf R \mathbf B  \mathbf R^T$ renders a diagonal matrix.
A: The coupled oscillators described this linear differential equations
$$\mathbf M\,\vec{\ddot{q}}+\mathbf K\,\vec{q}=\mathbf 0\tag 1$$
where $~\mathbf M~$ is the mass matrix and $~\mathbf K~$ is the stiffness matrix.
$$\mathbf M=\left[ \begin {array}{cc} m_{{1}}&0\\ 0&m_{{2}}
\end {array} \right] 
\\
\mathbf K=\left[ \begin {array}{cc} k_{{1}}+k&-k\\ -k&k_{{2}}
+k\end {array} \right] 
\\
\vec q=\left[ \begin {array}{c} x_{{1}}\\ x_{{2}}
\end {array} \right] 
$$
multiply equation (1) with $\mathbf M^{-1}$
$$\vec{\ddot{q}}+\underbrace{\mathbf M^{-1}\,\mathbf K}_{\mathbf A}\,\vec{q}=\mathbf 0\tag 2$$
with the eigenvalues $~\vec \lambda~$ of   matrix $~\mathbf A$ and the eigenvectors matrix $~\mathbf T~$  you can diagonalize the matrix  $~\mathbf A$
thus in modal space you get:
$$\vec{\ddot{q}}_m+\underbrace{\mathbf T^{-1}\,A\,\mathbf T}_{\vec \lambda}\,\vec q_m=\mathbf 0$$
the Hamiltonian of this equation is:
$$H=\frac 12 \vec{\dot{q}}_m\cdot\,\vec{\dot{q}}_m+
\frac 12 \vec q_m^T\,  
  \begin{bmatrix}
    \lambda_1 & 0 \\
    0 & \lambda_2 \\
  \end{bmatrix}\,\vec q_m$$
$$H=\frac 12\left(\dot x_{1m}^2+\dot x_{2m}^2\right) +\frac 12 \left(\lambda_1\,x_{1m}^2+\lambda_2\,x_{2m}^2\right)$$
Example:
assume that $~m_1=m_2=m~$ and $~k_1=k_2=k~$
the eigenvalues are $~\lambda_1=\frac{3k}{m}=\omega_1^2~,\lambda_2=\frac km=\omega_2^2~$
and the Hamiltonian
$$H=\frac 12\left(\dot x_{1m}^2+\dot x_{2m}^2\right) +\frac 12 \left(\omega_1^2\,x_{1m}^2+\omega_2^2\,x_{2m}^2\right)$$

equation (2):
$$\vec{\ddot{q}}+\underbrace{\mathbf M^{-1}\,\mathbf K}_{\mathbf A}\,\vec{q}=\mathbf 0$$
with $\vec q=\mathbf T\,\vec q_m$
$$\mathbf T\,\vec{\ddot{q}}_m+\mathbf A\,\mathbf T\,\vec q_m=\mathbf 0$$
$$\mathbf T^{-1}\mathbf T\,\vec{\ddot{q}}_m+\mathbf T^{-1}\,\mathbf A\,\mathbf T\,\vec q_m=\mathbf 0$$
$$\vec{\ddot{q}}_m+\mathbf\lambda\,\vec q_m=\mathbf 0$$
