How are unitary representations different from other representations? I understand that unitary representations arise naturally in quantum mechanics when groups act on the Hilbert space in a way that preserves probability.
I don't understand what details make unitary representations different from other representations. It seems as though physicists talk explicitly about "unitary representations" all the time.
Are there some theorems or examples that show why working with a "unitary" representation ought to be notable?
I've heard that compact Lie groups have only finite dimensional unitary representations, but seeing as so many important Lie groups in physics are non-compact this doesn't seem like the most vital reason.
 A: Since the Hamiltonian is hermitian, and the time evolution of a system is $U(t)=e^{-itH/\hbar}$, $U(t)$ is automatically unitary.  Moreover, unitary transformations play the role of rotations in 3d space, in the sense that they preserve the inner product:
$$
\langle \phi\vert \psi\rangle = \langle \phi'\vert\psi'\rangle\, ,\qquad
\vert\psi'\rangle=U\vert\psi\rangle\, ,\quad \vert\phi'\rangle=U\vert\phi\rangle
$$
and thus they preserve the physical predictions of quantum theory, which depends on $\vert \langle \phi\vert \psi\rangle\vert^2 = \vert \langle \phi'\vert \psi'\rangle\vert^2$.  This makes the predictions independent of the choice of initial basis vectors, much like the predictions of classical physics are independent of the initial choice of directions of the basis vectors.
This conclusion is applicable to compact or non-compact groups.
A: One property I like about unitary representations is that every invariant subspace has a complementary invariant subspace (namely the orthogonal complement).
From that it follows, that when we find an invariant subspace $V$ of a unitary representation, the representation can be decomposed into a direct sum of $V$ and its orthogonal complement.
Moreover, from that we see that every finite-dimensional unitary representation is a direct sum of irreducible representations.
A: For a compact Lie group, every representation can be made unitary.
Say you have a vector space $V$ and a group representation $\rho(g)$ which acts on $v \in V$.
Now say you have some hermitian inner product $\langle u, v \rangle$ for which $\rho$ is not a unitary representation, i.e.
$$
\langle \rho(g) u, \rho (g) v \rangle \neq \langle u, v \rangle.
$$
We can use this inner product to construct one for which $\rho$ is actually unitary. We just average over the inner product with Haar measure:
$$
\langle u ,v \rangle_{\rm new} = \int_G d g \langle\rho(g)u, \rho(g)v \rangle.
$$
Therefore, at least for compact Lie groups, any representation on a complex vector space can be considered to be unitary. The word "unitary" has no effect on the representation theory itsef.
