As you probably know, the Lie group of physical transformations of a quantum system acts on the Hilbert space of states of the system by means of a (strongly-continuous projective-) unitary representation of the group. $G \ni g \mapsto U_g$. This action is effective also on the observables of the system, represented by self-adjoint operators: The action of $g$ on the observables $A$ is $U_gAU^*_g$. The latter represents the observable $A$ after the action of the transformation $g$ on the physical system. This transformation has a twice intepretation. We can imagine that either it acts on the system or on the reference frame, our choice does not matter in this discussion.
Now let us focus on physics. There are natural elementary systems, called elementary particles. These systems are completely determined by fixing some real numbers corresponding to the values of some observables. Within the most elementary version of the story, these numbers are the mass $m$ which
may attain a few positive numbers experimentally observed ad recorded,
and the spin $s$ which may attain any number in $\{1/2, 1, 3/2,...\}$. Different values of the pair $(m,s)$ mean different particles.
These numbers have the property that they are invariant under the action of the most general symmetry group, I mean the (proper orthochronous) Poincaré group. A type of particle has the same fixed numbers $m$ and $s$ independently from the reference frame we use to describe it and the various reference frames are connected by the transformation of Poincaré group.
Passing to the theoretical quantum description of an elementary particle, in view of my initial remark, we are committed to suppose that its Hilbert space supports a representation of Poincaré group ${\cal P} \ni g \mapsto U_g$ (I omit technical details). Moreover there must be observables representing the mass $M$ and the spin $S$ that, on the one hand they must be invariant under the action of the group, i.e., $U_gM U_g^* =M$ and $U_gS U_g^* =S$ for every $g \in \cal P$. On the other hand they must assume fixed values $M=mI$ and $S=sI$.
Wigner noticed that a sufficient condition to assure the validity of these constraints is that ${\cal P} \ni g \mapsto U_g$ is irreducible.
Indeed, $M$ and $S$ can be defined using the self-adjoint generators of the representation, since they are elements of the universal enveloping algebra of the representation of the Lie algebra of $\cal P$ induced by the one of $\cal P$ itself. As expected, one finds $U_gM U_g^* =M$ and $U_gS U_g^* =S$ for every $g \in \cal P$. But, if $U$ is also irreducible, re-writing the identities above as
$U_gM =M U_g$ and $U_gS =S U_g$ for every $g \in \cal P$, Schur's lemma entails that $M=mI$ and $S=sI$ for some real numbers $s,m$.
To corroborate Wigner's idea it turns out that the two constants $m$ and $s$ are really sufficient to bijectively classify all possible strongly-continuous unitary irreducible representations of $\cal P$ with "positive energy" (the only relevant in physics).
The mathematical theory of representations of ${\cal P}$ autonomously fixes the possible values of $s$ and they just coincide with the observed ones. The values of $m$ are not fixed by the theory of representations where any value $m\geq 0$ would be possible in principle, though not all $m \geq 0$ correspond to the masses of observed elementary particles.
If you have many elementary particles, the Hilbert space of the system is the tensor product of the Hilbert spaces of the elementary particles and there is a corresponding unitary representation of Poincaré group given by the tensor product of the single irreducible representations. Obviously, the overall representation is not irreducible.
ADDENDUM. I would like to specify that the irreducible representations of the group of Poincaré I discussed above are the faithful ones whose squared mass is non-negative. Moreover, there is another parameter which classifies the irreducible representations of Poincaré group. It is a sign corresponding to the sign of energy. Finally not all particles fit into Wigner's picture.
