I am looking to reduce the dependence of a function, knowing that it satisfies some invariance constraints. Let me first formulate my question by explaining the 2-dimensional case.
Imagine I have a function (an Hamiltonian for example), that depends on two vectors in $\mathbb{R}^2$. Now suppose that I know this function is invariant under $SO(2)$ transformation on its vectors: $$ H(R(\theta)\vec{r}_1, R(\theta)\vec{r}_2) = H(\vec{r}_1, \vec{r}_2) $$ where $R(\theta)$ is a rotation matrix.
The problem I am trying to solve is to reduce the dependence of the hamiltonian to the minimal number of variables, which should 3 instead of the 4 variables $x_1, x_2, y_1, y_2$.
Intuitively, I know that the transform : $$ r_1 = \sqrt{x^2_1+y^2_1}\\ r_2 = \sqrt{x^2_2+y^2_2}\\ \phi_1 = \arctan(y_1/x_1) + \arctan(y_2/x_2) = \theta_1 + \theta_2\\ \phi_2 = \arctan(y_1/x_1) - \arctan(y_2/x_2) = \theta_1 - \theta_2 $$ will be the answer because the group action takes $\phi_1$ to $\phi_1+\theta$ and leaves the rest unchanged, which means that $H$ is independent of $\phi_1$.
Question: How to solve exactly the same problem for 3 vectors in $\mathbb{R}^3$ and with $H$ invariant with respect to $SO(3)$.
Bonus: Is there a natural generalization to higher dimensions?