Variables of an SO(3)-invariant function (hamiltonian) I am looking to reduce the dependence of a function, knowing that it satisfies some invariance constraints. Let me first formulate my question by explaining the 2-dimensional case.
Imagine I have a function (an Hamiltonian for example), that depends on two vectors in $\mathbb{R}^2$. Now suppose that I know this function is invariant under $SO(2)$ transformation on its vectors:
$$
H(R(\theta)\vec{r}_1, R(\theta)\vec{r}_2) = H(\vec{r}_1, \vec{r}_2)
$$
where $R(\theta)$ is a rotation matrix.
The problem I am trying to solve is to reduce the dependence of the hamiltonian to the minimal number of variables, which should 3 instead of the 4 variables $x_1, x_2, y_1, y_2$.
Intuitively, I know that the transform :
$$
r_1 = \sqrt{x^2_1+y^2_1}\\
r_2 = \sqrt{x^2_2+y^2_2}\\
\phi_1 = \arctan(y_1/x_1) + \arctan(y_2/x_2) = \theta_1 + \theta_2\\
\phi_2 = \arctan(y_1/x_1) - \arctan(y_2/x_2) = \theta_1 - \theta_2
$$
will be the answer because the group action takes $\phi_1$ to $\phi_1+\theta$ and leaves the rest unchanged, which means that $H$ is independent of $\phi_1$.
Question: How to solve exactly the same problem for 3 vectors in $\mathbb{R}^3$ and with $H$ invariant with respect to $SO(3)$.
Bonus: Is there a natural generalization to higher dimensions?
 A: Think of a triangular pyramid, a tetrahedron. When you rotate it in space, it does not alter. The three vectors you are given, $\vec r_1, \vec r_2, \vec r_3$, are the edges with an ant on them. 

It should be evident that you may completely characterize the solid by the length of three anted edges, and the three angles of the corresponding faces; 6 variables in all: $r_1,r_2,r_3,\vec r_1\cdot \vec r_2,\vec r_2\cdot \vec r_3, \vec r_3\cdot \vec r_1 $.
Alternatively, by the lengths of all 6 edges, $\vec r_1, \vec r_2, \vec r_3,\vec r_1- \vec r_2, \vec r_2 -\vec r_3, \vec r_3- \vec r_1$. You may see that the 6 lengths involved allow you to determine the three cosines of the angles above instead. So 6 lengths are equivalent to 3 lengths and 3 angles. 
Bonus: Assuming you add a new vector for each dimension, you note that in 4 dimensions you have 4 vectors and 6 angles, so 10 d.o.f.s. Consequently, mutatis mutandis, in d dimensions, you have d vectors and ${d\choose 2}=d(d-1)/2  $ angles, so $d(d+1)/2$ d.o.f.s in all...
