Why $3$-dim isotropic harmonic oscillator's symmetry is not $O(6)$ but $U(3)$?

Question

3D harmonic oscillator's Hamiltonian is $$H=\sum_{i=1}^3p_i^2+q_i^2$$

Why all textbooks say that its symmetry is $U(3)$. But I think it's $O(6)$. Because the rotation of $6$ coordinates in phase space is invariant.

Answer 1

The statement holds both in the classical and the quantum case:

Clasical mechanics

The symmetry group of the function $H:\mathbb{R}^3\times\mathbb{R}^3\rightarrow\mathbb{R}$ given by $H(q,p)=\sum_{i=1}^3(q_i^2+p_i^2)$ is indeed $O(6)$, as you guessed. However, when talking about symmetries an extra condition is needed: they should preserve the symplectic structure.

To understand what is meant by this, remember that Hamilton's equations are $\dot{q_i}=\partial H/\partial p_i$ and $\dot{p_i}=-\partial H/\partial q_i$. They may be written as \begin{equation} \left(\begin{array}{c}\dot{q}\\\dot{p}\end{array}\right)= \left(\begin{array}{cc}0 & I \\ -I & 0\end{array}\right) \left(\begin{array}{cc}\frac{\partial H}{\partial q} \\ \frac{\partial H}{\partial p}\end{array}\right) \end{equation} where $I$ is the $3\times 3$ identity matrix. Thus, if we want the form of Hamilton's equations to be invariant under a transformation \begin{equation} \left(\begin{array}{c}q\\p\end{array}\right)\mapsto M\left(\begin{array}{cc}q \\ p\end{array}\right) \end{equation} it should satisfy $M^T \Omega M = \Omega$ where \begin{equation} \Omega = \left(\begin{array}{cc}0 & I \\ -I & 0\end{array}\right). \end{equation}

The group of such matrices $M$ is the symplectic group $Sp(6,\mathbb{R})$. The group of symmetries should now be the intersection of $O(6)$ and $Sp(6,\mathbb{R})$. Using the general property $U(n)=O(2n)\cap Sp(2n,\mathbb{R})$ we get that the group we are looking for is $U(3)$.

Quantum mechanics

An explanation of why the symmetry group can't be $O(6)$ (in a more general case) can be found in the answer to this question, as pointed out in the comments.

Now we want to see that the symmetry group is $U(3)$. We require the invariance of $H$ and of commutation relations $[q_i,q_j]=[p_i,p_j]=0$, $[q_i,p_j]=i\delta_{ij}$, which can be written collectively as $[(q,p)^T,(q,p)]=i\Omega$. Therefore the commutator preserving transformations are $Sp(2,\mathbb{R})$ and by the same reasoning as before the group of symmetries is $U(3)$.

Answer 2

The canonical references for this are:

1. Fradkin, D. M. "Three-dimensional isotropic harmonic oscillator and SU3." American Journal of Physics 33.3 (1965): 207-211,
2. Harvey, Malcolm. "The Nuclear SU3 Model." Advances in nuclear physics. Springer US, 1968. 67-182

In short, the symmetry group of the $n$-dimensional harmonic oscillator is best understood by expressing the momenta and positions using the creation and destruction operators $\hat a_k^\dagger$ and $\hat a_k$, with $k=1,\ldots,n$. This makes it clear that the symmetry group is not limited to real transformations and to "point" transformations mixing positions and moments separately.

Any complex $n\times n$ transformation $U$ sending $\hat a_k\to U\hat a_k$ must also send $\hat a_k^\dagger\to \hat a_k^\dagger U^\dagger$. Thus $H=\sum_k \hat a_k^\dagger \hat a_k$ invariant if $U^\dagger U$ is the unit matrix, whence $U$ is an $n\times n$ unitary matrix.