Addition of Two Elements of Group Representation (Quantum Mechanics angular momentum) In Sakurai's Modern Quantum Mechanics I saw the author takes commutation of two infinitesimal 3D rotation matrices. He also claims that the Hilbert space rotation operators should satisfy the same commutation relation.

*

*What does it mean to subtract two elements in the representation of a group, namely, lets say $R(ε)$ to be infinitesimal matrix by infinitesimal angle $ε$ then what does
$$
R_x(ε)R_y(ε) - R_y(ε)R_x(ε) = R_z(ε^2)-1
$$
mean? How can we subtract rotations from each other because rotations form a group in which multiplication operation is defined (applying rotations consecutively). But we do not have any addition operation in rotation group, so how do we have such an operation in the representation of the group and what does it mean such things like the infinitesimal rotation commutations are related to second order infinitesimal rotations around some other axis.


*He also claims that the Hilbert space rotation operators should satisfy the same commutation relation. How can he be sure about this statement
 A: You appear to know a bit more about the rotation group than the authors assume: They start from basic infinitesimal rotations in three dimensions, standard orthogonal matrices, and steer you in unravelling their Lie group structure in elementary terms.

*

*Remember, the group commutator involves only multiplications/inversions, but the algebra commutator involves those and also additions/subtractions.

A Lie group sophisticate understands, instead,
$$
R_y(\epsilon)= e^{\epsilon L_y}= {\mathbb I} + \epsilon L_y + O(\epsilon^2), \\  L_y\equiv 
\begin{pmatrix} 0&0& 1 \\
0&0&0\\ -1&0&0 \\ \end{pmatrix}= -L_y^T,
$$
etc, for the other two directions.
So a rotation, a group element, is the identity plus a generator, plus smaller higher order terms.
The group commutator of two group elements, $R_x(\epsilon), R_y(\epsilon)$, then, is defined  as the succession of rotations, so a group element,
$$
R^{-1}_x(\epsilon)R^{-1}_y(\epsilon)R_x(\epsilon)R_y(\epsilon)\\ \approx ({\mathbb I} - \epsilon L_x +...)({\mathbb I} - \epsilon L_y +...)({\mathbb I} + \epsilon L_x +...)({\mathbb I} + \epsilon L_y +...)\\ 
\approx ({\mathbb I} - \epsilon L_x   - \epsilon L_y +\epsilon^2[L_x,L_y]/2+...)({\mathbb I} + \epsilon L_x +\epsilon L_y +\epsilon^2[L_x,L_y]/2+...)\\
={\mathbb I}+\epsilon^2 [L_x,L_y] + O(\epsilon^3) \approx 
R_z(\epsilon^2) \\ \approx {\mathbb I}+ R_x(\epsilon)R_y(\epsilon)-R_y(\epsilon)R_x(\epsilon)  .
$$
N.B. Recall the CBH expansion,
$e^{\epsilon L_x}e^{\epsilon L_y}= e^{\epsilon L_x+\epsilon L_y+\epsilon^2 [L_x,L_y]/2+...}$. So, to $O(\epsilon^2)$, the group commutator reduces to the identity,  corrected to a smaller rotation at  $\epsilon^2$.

*

*At order $\epsilon^2$, infinitesimal rotations reconstruct the Lie algebra of generators, (3.9),
$$[R_x(\epsilon)-{\mathbb I}, R_y(\epsilon)-{\mathbb I}]= R_z(\epsilon^2)-{\mathbb I}   
~\leadsto ~ [L_x,L_y]=L_z.$$
That is, the authors introduce the Lie algebra  commutator without  explicitly introducing the Lie algebra, or its exponential,  merely through infinitesimal rotation sequences, and addition of group elements (!) approximating the well-defined group commutator. The addition is a feature of the algebra, not the group, but is well defined for matrices, as utilized here. Neat, huh?
Reading on will connect you to this underlying Lie algebra.
If you already  appreciate the magic of the Lie structure, the book's illustration is reassuring or distracting overkill; but if you don't, it is as good as a bona-fide seat-of-the pants introduction, quite sly for an introductory physics text―the hallmark of JunJohn S.
Once you have appreciated this underlying group theory, the text eases you into the most general representations suitable to  your Hilbert space, all subject to the same Lie algebraic, and thus Lie group symmetry constraints! The combinatorics involved here in the CBH expansion are identical to those in every and any representation.
A: A group representation is, by definition, a homomorphism from the group into the group of linear operators on some vector space $V$. In less technical terms, a representation is a way of writing the group as matrices acting on some vector space, with the product operation being given by the usual matrix multiplication.
However, matrices always admit addition and subtraction. Hence, in the representation you do happen to have a natural notion of addition and subtraction. In the abstract group, this wouldn't be possible, but it is in the representation because it has extra structure.
It is also worth pointing out that he is adding and subtracting infinitesimal rotations. While that might seem innocent, it means he isn't really working with the group elements. Instead, he is working with something known as the Lie algebra associated to the group. The Lie algebra is, intuitively, formed by infinitesimal elements of the group and it does have a vector space structure, so one can add and subtract elements of the Lie algebra. I'm not sure if Sakurai uses these sorts of construction explicitly in his text, but I'm mentioning it in here for completeness. A good introduction to these themes is A. Zee's Group Theory in a Nutshell for Physicists.
As for your second question, notice the Hilbert space is a vector space, so the rotation operators are actually just a representation of the group. If they were not a representation, you could hardly call them "rotation": rotations are defined as elements of the rotation group, so the rotation operators must be implementing the notions of that group in the Hilbert space. Now, the result Sakurai obtained for the commutator of infinitesimal rotations holds for any representation, so it holds in particular for the representation you're using in the Hilbert space.
