Representation on Hilbert space of the product of two symmetry transformations We know by Wigner's theorem that the representation of a symmetry transformation on the Hilbert space is either unitary and linear, or anti-unitary and anti-linear.
Let $T$ and $S$ be two symmetry transformations. Let $U(T)$ and $U(S)$ be the representations of these transformations. What can we say about unitarity or anti-unitarity of $U(TS)$ if we know the unitarity or anti-unitarity of U(T) and $U(S)$? Why?
 A: I) Wigner's Theorem states that a symmetry operation $S: H \to H$ is a unitary or anti-unitary$^{1}$ operator $U(S)$ up to a phase factor $\varphi(S,x)$,
$$ S(x)~=~ \varphi(S,x)\cdot U(S)(x), \qquad x~\in~H,\qquad \varphi(S,x)~\in~\mathbb{C} ,\qquad |\varphi(S,x)|~=~1 .$$
In this context, a symmetry operation $S$ is by definition a surjective (not necessarily linear!) map $S: H \to H$ such that 
$$|\langle S(x),S(y)\rangle|~=~|\langle x,y\rangle|,\qquad\qquad x,y~\in~H.$$
Let us introduce the terminology that a symmetry operation $S$ is of unitary (anti-unitary) type if there exists a unitary (an anti-unitary) $U(S)$, respectively.
Moreover, if ${\rm dim}_{\mathbb{C}} H \geq 2$, then one may show that


*

*$U(S)$ is unique up to a constant phase factor, and 

*$S$ cannot have both a unitary and an antiunitary $U(S)$. In other words, $S$ cannot  both be of unitary and anti-unitary type.


II) It follows by straightforwardly applying the definitions, that the composition $S \circ T$ of two symmetry operations $S$ and $T$ is again a symmetry operation, and it is even possible to choose
$$ U(S \circ T)~:=~U(S) \circ U(T).$$
Finally, in the case ${\rm dim}_{\mathbb{C}} H \geq 2$,


*

*$S \circ T$ is of anti-unitary type, if precisely one of $S$ and $T$ are of anti-unitary type, and 

*$S \circ T$ is of unitary type, if zero or two of $S$ and $T$ are of anti-unitary type. 


Reference:


*

*V. Bargmann, Note on Wigner's Theorem on Symmetry Operations, J. Math. Phys. 5 (1964) 862. Here is a link to the pdf file.


--
$^{1}$ We use for convenience a terminology where linearity (anti-linearity) of $U(S)$ are implicitly implied by the definition of $U(S)$ being unitary (anti-unitary), respectively. 
