A discussion about $\text{SO}(1,3)$ transformation generators on states and four-momentum little group The question is quite long, this because many topics are touched, but I tried to make it the more self-contained I could: on the other side, many arguments are touched also to make it self-contained.
I really love to discuss, so please if you have some free time to read it, see it more like a relaxed discussion than a question.
I start from Lorentz transformation on a field, I will quickly pass on four-momentum states and I will arrive to the corresponding little group and so my question
$\text{SO}(1,3)$ transformation generators for a field
The generator of an $\text{SO}(1,3)$ field transformation has the following form
$$
{\mathfrak{J}^{\gamma\delta\alpha}}_\beta
=
{\Sigma^{\gamma\delta\alpha}}_\beta
+
{\mathbb{I}^\alpha}_\beta
\left(
x^\gamma
\partial^\delta
-
x^\delta
\partial^\gamma
\right)
$$
where ${\Sigma^{\gamma\delta\alpha}}_\beta$ is the generic component of $\text{SO}(1,3)$ transformation generator acting on the internal components of the field, indexed in $\alpha,\beta$ (while as it's clear from the formula $\gamma,\delta$ are referring to the component of the $\boldsymbol{x}$ position index); the form ${\Sigma^{\gamma\delta\alpha}}_\beta$ is not given because it depends on the kind of field, but there is one thing that is known: in its various representations should obey the commutation relation of all the $\text{SO}(1,3)$ generators
$$
\left[
\Sigma^{\gamma\delta},
\Sigma^{\zeta\eta}
\right]
=
\mathbb{G}^{\gamma\eta}
\Sigma^{\delta\zeta}
+
\mathbb{G}^{\delta\zeta}
\Sigma^{\gamma\eta}
+
\mathbb{G}^{\zeta\gamma}
\Sigma^{\eta\delta}
+
\mathbb{G}^{\eta\delta}
\Sigma^{\zeta\gamma}
$$
$\text{SO}(1,3)$ transformation generators for four-momentum eigenstate
That said, consider the generic $\text{SO}(1,3)$ transformation on the generic four-momentum eigenstate
$$
e^{\frac{i}{\hbar}\vartheta^\alpha \hat{\mathfrak{J}}_\alpha}
|\boldsymbol{p},\mu\rangle
=
{\hat{\Sigma}(\boldsymbol{\vartheta})^\mu}_\nu
|\Lambda(\boldsymbol{\vartheta})\boldsymbol{p},\nu\rangle
$$
This kind of transformation is very very strange to me if I try to think $|\boldsymbol{p},\mu\rangle$ as the close relative of a field.
I don't understand if this last given formula is a definition, a request or a consequence of some principle I'm not considering.
Little group of the four-momentum in the massive case
But that was not the only question I have: in the situation of a massive "particle" I should choose a reference such that the fourmomentum eigenstate is simply $|(mc,\boldsymbol{0}),\mu\rangle$ (or instead of talking about particles and reference should I just talk about fundamental state of the momentum?) and in that case can be demonstrated that the little group of the momentum is simply $\text{SO}(3)$ (is actually $\text{SU}(2)$, but in the book I'm reading it pretends is just a tridimensional rotation, to be able to do the following).
We know that $\text{SO}(3)$ generators $\mathbb{J}_i$ are related to the generic $\mathbb{J}^{\alpha\beta}$ Lorentz generators through
$$
\mathbb{J}_i
=
\frac{1}{2}\epsilon_{ijk}\mathbb{J}^{jk}
$$
and so, apparently just comparing to this last formula, I should say that the $\text{SO}(3)$ generators on the states are just
$$
\hat{\mathfrak{J}}_i
=
\frac{1}{2}\epsilon_{ijk}\hat{\mathfrak{J}}^{jk}
$$
so that defining the spin operator as
$$
\hat{\Sigma}_i
\doteq
\frac{1}{2}
\epsilon_{ijk}
\hat{\Sigma}^{jk}
$$
I should conclude that
$$
\hat{\mathfrak{J}}_i
=
\hat{\Sigma}_i
-
\frac{i}{\hbar}
\epsilon_{ijk}
x^j
\hat{p}^k
$$
where $\hat{p}^k=i\hbar\partial^k$ are the spatial components of the four-momentum operator. These are the generators of the little group transformation for the four-momentum in the form $|(mc,\boldsymbol{0}),\mu\rangle$. Even if I'm not completely sure, this seems to mean that the infinitesimal little group transformation has this "field" operator
$$
{\hat{\Sigma}(\text{d}\boldsymbol{\vartheta})^\mu}_\nu
=
{(1+\text{d}\vartheta^i\hat{\mathfrak{J}}_i)^\mu}_\nu
$$
So that now I finally found the explicit expression for the infinitesimal little group transformation on the state and is the following
$$
e^{\frac{i}{\hbar}\text{d}\vartheta^\alpha\hat{\mathfrak{J}}_\alpha}
|(mc,\boldsymbol{0}),\mu\rangle
=
{\left(1+\text{d}\vartheta^i\left(\hat{\Sigma}_i
-
\frac{i}{\hbar}
\epsilon_{ijk}
x^j
\hat{p}^k\right)\right)^\mu}_\nu
|(mc,\boldsymbol{0}),\nu\rangle
$$
The momentum operator acting on that state is clearly null in all spatial components, meaning that $\hat{p}^k|(mc,\boldsymbol{0}),\mu\rangle=0$ and I remain with
$$
e^{\frac{i}{\hbar}\text{d}\vartheta^\alpha\hat{\mathfrak{J}}_\alpha}
|(mc,\boldsymbol{0}),\mu\rangle
=
|(mc,\boldsymbol{0}),\mu\rangle
+\text{d}\vartheta^i{{{\hat{\Sigma}_i}^\mu}_\nu}
|(mc,\boldsymbol{0}),\nu\rangle
$$
Finally, the question
(You are my hero if you arrived until here)
I understand that a little group transformation of the $|(mc,\boldsymbol{0}),\mu\rangle$ state should lead to a superposition of states only on the form $|(mc,\boldsymbol{0}),\nu\rangle$, but how can I pass from this consideration to affirming that these are actually the eigenstates of $\hat{\Sigma}_i$? And what are the role of the $\mu,\nu$ indexes in ${{{\hat{\Sigma}_i}^\mu}_\nu}$? Because they mess up my conception of $\hat{\Sigma}_i$ as an operator acting on states, if it implies a summation also on them (for example in the Pauli-Lubanski four-vector $\hat{\Sigma}_i$ appears too, but without $\mu,\nu$ indexes, because Pauli-Lubanski four-vector acts on one state and not on a summation of them).
I would be so grateful to have just a little diriment answer. I'm so confused. Thank a lot!
 A: Let me make some statements which may help organize things a bit. Explicit forms for the Lorentz transformations on the states are not necessary to see what's going on with the little group, and may obfuscate things a bit.
Let $P^\mu$ be our 4-momentum operator and suppose that our states are written $|p,\sigma\rangle$ where $\sigma$ is any index needed for these vectors to span the Hilbert space. I will assume that $\sigma$ is discrete (I think this requirement can be relaxed, but would make things technically more complex).
By definition, $P^\mu|p,\sigma\rangle=p^\mu|p,\sigma\rangle$. If we suppose that $U(\Lambda)$ is the unitary operator implementing Lorentz transformations on our Hilbert space, then it forms a representation, $U(\Lambda)U(\Lambda^\prime)=U(\Lambda\Lambda^\prime)$, and must map a $p$-momentum state to a $\Lambda p$-momentum state.
This can be explicitly shown from the commutator algebra with the momentum operator, but in the end we have
$$
P^\mu(U(\Lambda)|p,\sigma\rangle)=(\Lambda^\mu_\nu p^\nu)(U(\Lambda)|p,\sigma\rangle).
$$
From this it follows that $U(\Lambda)|p,\sigma\rangle$ must be some linear combination of eigenvectors of $P^\mu$, all with eigenvalue $\Lambda^\mu_\nu p^\nu$. Necessarily then there exist coefficients $C_{\sigma,\sigma^\prime}(\Lambda,p)$ such that
$$
U(\Lambda)|p,\sigma\rangle=\sum_{\sigma^\prime}C_{\sigma,\sigma^\prime}(\Lambda,p)|\Lambda p,\sigma^\prime\rangle.
$$
I have made no statements about what these coefficients are yet, so this is the most general possible result for the action of a Lorentz transformation on a state.
Now that some notation has been established, my goal will be to show two things: these coefficients must form a representation of the little group, and that the little group for a massive particle must be SO(3), and hence diagonalizing the little group representations is equivalent to decomposing the system into spin representations.
Denoting the standard momenta $k^\mu=(mc,\boldsymbol 0)$ (the following also works for standard momenta $(1,-1,0,0)$), I will generally write $W$ to indicate an element of the little group, satisfying $W^\mu_\nu k^\nu=k^\mu$. We can also define the special Lorentz transformation $L(p)$ which maps from the standard momenta to any given momenta $p$.
Now, since the $\sigma$'s are just some indices, we have some freedom in which state we define $|p,\sigma\rangle$ to represent. In particular, we may define the symbol $|p,\sigma\rangle$ by $|p,\sigma\rangle=U(L(p))|k,\sigma\rangle$ (this can probably be though analogous to the statement that given $\boldsymbol{\hat x}$ in $\mathbb{R}^3$, there is some freedom in what we call $\boldsymbol{\hat y}$ and $\boldsymbol{\hat z}$ if our only demands are they be orthogonal to each other and form a right-handed coordinate system). There are some fine points about choosing state normalizations to make sure the normalizations themselves remain Poincare invariant, but they will not matter for what I would like to say here.
With this, we are able to write
$$
U(\Lambda)|p,\sigma\rangle=U(\Lambda L(p))|k,\sigma\rangle=U(L(\Lambda p))U(L^{-1}(\Lambda p)\Lambda L(p))|k,\sigma\rangle\tag{1}
$$
by inserting the identity. If we stare at $W(\Lambda,p)\equiv L^{-1}(\Lambda p)\Lambda L(p)$ for long enough, we may also note that this is a little group element since it maps $k\rightarrow p\rightarrow \Lambda p\rightarrow k$.
For general little group elements $W$ we have, for some coefficients $D_{\sigma\sigma^\prime}(W)$, $U(W)|k,\sigma\rangle=\sum_{\sigma^\prime}D_{\sigma\sigma^\prime}(W)|k,\sigma^\prime\rangle$ as a special case of the general statement about the action of Lorentz transformations on our states. In particular, $U(W^\prime)U(W)=U(W^\prime W)$ implies that these coefficients satisfy
$$
D_{\sigma\sigma^\prime}(W^\prime W)=\sum_{\sigma^{\prime\prime}}D_{\sigma\sigma^{\prime\prime}}(W^\prime)D_{\sigma^{\prime\prime}\sigma^\prime}(W)\tag{2}
$$
and hence form a representation of the little group.
Combining (1) and (2), we are now able to write
$$
U(\Lambda)|p,\sigma\rangle=U(L(\Lambda p))\sum_{\sigma^\prime}D_{\sigma\sigma^\prime}(W(\Lambda, p))|k,\sigma^\prime\rangle=\sum_{\sigma^\prime}D_{\sigma\sigma^\prime}(W(\Lambda, p))|\Lambda p,\sigma^\prime\rangle.\tag{3}
$$
Hence, we have found $C_{\sigma\sigma^\prime}(\Lambda,p)=D_{\sigma\sigma^\prime}(W(\Lambda,p))$, the coefficients $D$ already shown to form a representation of the little group. With this, the spectrum of states may be further decomposed into irreducible representations of the little group.
In the case of a massive particle, $k^\mu=(mc,\boldsymbol{0})$, so the little group is just SO(3). Hence the irreducible representations of the little group that our states should be organized by will in general be spin. So it's not that you can ensure the states will be eigenstates of your $\hat \Sigma$ operator, but rather that the states can always be reorganized such that this is true.
The arguments I have given above are very similar to those in chapter 2 of Weinberg's The Quantum Theory of Fields Volume I (where I learned this from). More details can be found there.
Based on a discussion in the comments, I feel there are a few more things I should add to this answer. First, I would like to point out that the main result of this argument was to show that the coefficients $C_{\sigma\sigma^\prime}(\Lambda, p)$, whose existence is guaranteed on general grounds, can be expressed in terms of the coefficients $D_{\sigma\sigma^\prime}(W)$ which were noted to form a representation of the little group. In particular, (3) implies (up to some normalizations which don't matter here but are worked out in Weinberg)
$$
C_{\sigma\sigma^\prime}(\Lambda,p)=D_{\sigma\sigma^\prime}(W(\Lambda,p)),
$$
and so the $C$'s also form a representation of the little group. A fairly accessible description of representation theory can be found in sections 1.1-1.5 of Howard Georgi's Lie Algebras in Particle Physics.
For a massive particle, the little group was noted to be $SO(3)$, so let's discuss the relationship between $SO(3)$ and spin, which requires we first talk about $SU(2)$. Everything I am about to describe can be found in chapters 2 and 3 of Georgi's book, mentioned above. There are three generators which span the Lie algebra of $SU(2)$. At least some of this should be familiar from standard accounts of spin $1/2$, though the generalization to higher spin is typically left out. In any case, all the finite dimensional representations of $SU(2)$ can be classified by the spin, which is the largest eigenvalue of the diagonalized generator (typically $S_z$ by convention).
So, whenever someone refers to some state or some such as being "spin-$j$", the meaning is that this object transforms under the representation of $SU(2)$ with spin $j$. The dimensionality of the spin-$j$ representation is always $2j+1$, so for $j=1/2$ the dimension is $2$, which we know had to be the case because the Pauli matrices are used when discussing spin-$1/2$ particles/states.
This index $\sigma$ runs over the dimensionality of the representation. So in some sense it's misleading to call $\sigma$ "spin" when spin is understood to have the meaning above.
Relating this back to the case where our little group is $SO(3)$, note that the Lie algebras of $SO(3)$ and $SU(2)$ are isomorphic, so all the statements about representations of $SU(2)$ carry over to the case of $SO(3)$. There are some subtleties relating to global properties of the two groups, but I think worrying about these details would distract from the main idea here.
We can go even further to explore how these $C$'s work, what these $\sigma$ indices are supposed to be, and how this relates to the $C$'s forming a representation.
A large majority of what I said above about $SU(2)$ really only applies to so-called irreducible representations of the group. It is a theorem that all representations of any finite group can be decomposed into the direct sum of irreducible representations. In fact, this is precisely what we are doing whenever we compute a Clebsch-Gordon decomposition.
With this in mind, the $C$'s don't need to form an irreducible representation (which are the things actually classified by the spin $j$), but we can always rotate our basis vectors such that the $C$'s have been block diagonalized, and hence we have decomposed our states into sectors transforming under irreducible representations of $SU(2)$. These sectors are typically what we would identify as "spin-$j$ particles."
Within a single sector, say for simplicity a spin-$1/2$ block, our states are spanned by the states $|p;1/2;\sigma_{1/2}\rangle$ where now because we are within a single block (subspace) of the Hilbert space, the $\sigma$ runs over the indices of the spin-$1/2$ representation. This is a two-dimensional representation, so $\sigma_{1/2}=1,2$, or more colloquially, is either spin up or spin down.
A: Why this answer
I noticed that I had much difficulty in understanding the very valid and useful answer given here, because I did not understand well how little groups worked. I hope with this integration to make things clearer for the people that in the future will need it, and I hope not to make things worse. Please let me know, or edit, if something is imprecise, unclear or wrong.
What are we looking for
We are constructing a field theory in an attempt to describe particles, with properties that must not change if we apply a certain transformation: we will see in particular $\text{O}(1,3)$ scalars, numbers that don't change when we ''rotate'' in Minkowski spacetime; for example, the rest mass $m$ of the electron doesn't change if I rotate or apply a boost in spacetime and the same for its spin $s$.
Groups and representations
Let's talk about groups.
I put my hands on an abstract group $\mathscr{G}$ representing it trough matrices, doing an homomorphism $\varphi$ between it and a subgroup $\mathfrak{M}_\mathscr{G}\subset\text{GL}(m)$ of the general linear group, with $m$ called the order of the representation. If $\varphi$ is an isomorphism the representation is said to be faithful.

$\text{O}(3)$ group is called this way by taking this argument the opposite way: its fundamental representation has dimension $m=3$ with every matrix satisfying $\mathbb{M}^{\text{T}}\mathbb{I}\mathbb{M}=\mathbb{I}$, so they are orthogonal matrices (in euclidean metric) of order three.
I can transpose the $\text{O}(3)$ transformation in $4$ dimensions and clearly the representation of $\text{O}(3)$ won't anymore be fundamental (the matrices of the group representation will now have order $4$). The ordered $m$-dimensional space $\mathfrak{V}$ in which these $m$-order square matrices act is the carrier or representation space; in the fundamental representation of $\text{O}(3)$ we have $m=3$.
Analogously we define rotations $\text{O}(1,3)$ in Minkowski metric (diagonal metric tensor $\eta=(1,-1,-1,-1)$) satisfying $\mathbb{M}^{\text{T}}\eta\mathbb{M}=\eta$ in their fundamental representation.
Little groups
Our field configuration state $|\boldsymbol{v}\rangle$, carrying all the informations about the field in space and time, can be decomposed in $|\boldsymbol{x}\rangle\equiv|t,\boldsymbol{r}\rangle$ or $|\boldsymbol{p}\rangle$ bases through closure relations, so consider this last case. Consider one single momentum eigenstate $|\boldsymbol{p}\rangle$ such that
\begin{gather*}
\hat{\boldsymbol{p}}|\boldsymbol{p}\rangle\doteq \boldsymbol{p}|\boldsymbol{p}\rangle
\end{gather*}
The concept behind little groups is that I can Lorentz-transform non-trivially a four-vector, in this discussion the eigenvalue $\boldsymbol{p}$, and leave it invariant
\begin{gather*}
\exists\,\mathbb{M}(\boldsymbol{p})\neq\mathbb{I}:\mathbb{M}(\boldsymbol{p})\boldsymbol{p}=\boldsymbol{p}%,\,
%\mathbb{M}(\boldsymbol{p})\in\varphi\left(\text{O}(1,3)\right)
\end{gather*}
and I stress the fact that $\mathbb{M}(\boldsymbol{p})$ is a matrix of $\text{O}(1,3)$ (in its fundamental representation because $\boldsymbol{p}$ is a Lorentz four-vector by definition)
Expliciting little group transformations
The explicit form of the matrix depends on the particular $\boldsymbol{p}$ on the table, but we are able to choose one representative $\tilde{\boldsymbol{p}}$ for every one of the following classes

*

*$p^\alpha p_\alpha>0$

*$\boldsymbol{p}\neq\boldsymbol{0},p^\alpha p_\alpha\neq 0$

*$\boldsymbol{p}=\boldsymbol{0}$ just him

*$p^\alpha p_\alpha<0$
The reason is that choosing a class-representative $\tilde{\boldsymbol{p}}$ I can find an $\mathbb{M}(\tilde{\boldsymbol{p}})$ that is particularly easy to calculate, and from that derive all the $\mathbb{M}(\boldsymbol{p})$ of the class, making them explicit in their $\boldsymbol{p}$-dependence.
How can we do this?
Consider the first class of $p^\alpha p_\alpha>0$, four-momentum of a ''massive particle'' with undetermined mass; we can always find a rest-reference frame $\tilde{\boldsymbol{p}}=(mc,0,0,0)$ from which we can $\text{O}(1,3)$-rotate and go in every possible other frame
\begin{gather*}
\boldsymbol{p}
\equiv
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
\tilde{\boldsymbol{p}}
\end{gather*}
The generic little group equation now becomes
\begin{gather*}
\mathbb{M}(\boldsymbol{p})
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
\tilde{\boldsymbol{p}}
=
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
\tilde{\boldsymbol{p}}
\\
\left(\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})\right)^{-1}
\mathbb{M}(\boldsymbol{p})
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
\tilde{\boldsymbol{p}}
=
\tilde{\boldsymbol{p}}
\end{gather*}
Magic trick! Because I invite you to look closely to this last equation, you can conclude that
\begin{gather*}
\left(\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})\right)^{-1}
\mathbb{M}(\boldsymbol{p})
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
=
\mathbb{M}(\tilde{\boldsymbol{p}})
\end{gather*}
that should actually be an inclusion relation, but it can be demonstrated it's an equality: rewriting it
\begin{gather*}
\mathbb{M}(\boldsymbol{p})
=
\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})
\mathbb{M}(\tilde{\boldsymbol{p}})
\left(\mathbb{L}(\boldsymbol{p},\tilde{\boldsymbol{p}})\right)^{-1}
\end{gather*}
this is the promise of before: from $\tilde{\boldsymbol{p}}$ representative we find a simple $\mathbb{M}(\tilde{\boldsymbol{p}})$ by explicitily solving the system
\begin{gather*}
 \begin{cases}
\mathbb{M}(\tilde{\boldsymbol{p}})\tilde{\boldsymbol{p}}=\tilde{\boldsymbol{p}}
\\
\mathbb{M}^{\text{T}}(\tilde{\boldsymbol{p}})
\eta
\mathbb{M}(\tilde{\boldsymbol{p}})
=
\eta
 \end{cases}
\end{gather*}
from that we can recover every $\mathbb{M}(\boldsymbol{p})$
A first try for little group transformations on states
That was about the eigenvalues $\boldsymbol{p}$ of states $|\boldsymbol{p}\rangle$, but we want to come back to states; we first note that
\begin{gather*}
\hat{\boldsymbol{p}}
|\boldsymbol{p}\rangle
=
\boldsymbol{p}
|\boldsymbol{p}\rangle
=
\mathbb{M}(\boldsymbol{p})
\boldsymbol{p}
|\boldsymbol{p}\rangle
=
\mathbb{M}(\boldsymbol{p})
\boldsymbol{p}
|\mathbb{M}(\boldsymbol{p})\boldsymbol{p}\rangle
=
\boldsymbol{p}
|\mathbb{M}(\boldsymbol{p})\boldsymbol{p}\rangle
\end{gather*}
you got this, you can play around as long as you want, but what about making this little group transformation an operator? You can see, it's still a matrix $\mathbb{M}(\boldsymbol{p})$ we are talking about. To do this we consider a representation of the $\text{O}(1,3)$ group to which the $\mathbb{M}(\boldsymbol{p})$ belongs, in the carrier space of the states; this representation is called a realization because acts on an infinite-dimensional carrier space, and we can describe its general element like $\hat{\varphi}(\mathbb{L})$. We can imagine that a realization of the little group should act like
\begin{gather*}
\hat{\varphi}\left(\mathbb{M}(\boldsymbol{p})\right)
|\boldsymbol{p}\rangle
=
|\boldsymbol{p}\rangle
\end{gather*}
but stated this way creates a problem; if you consider an internal product
\begin{gather*}
\langle\boldsymbol{p}_1|
\hat{\varphi}\left(\mathbb{M}(\boldsymbol{p})\right)
|\boldsymbol{p}\rangle
=
\langle\boldsymbol{p}_1|\boldsymbol{p}\rangle
\\
\to
\langle\boldsymbol{p}_1|
\hat{\varphi}\left(\mathbb{M}(\boldsymbol{p})\right)
|\boldsymbol{p}\rangle
\propto
\delta(\boldsymbol{p}_1-\boldsymbol{p})
\end{gather*}
the operator $\hat{\varphi}\left(\mathbb{M}(\boldsymbol{p})\right)$ acts non-trivially on $\langle\boldsymbol{p}_1|$ so the equality is not true.
Little group transformations on states
To define the correct little group transformation on states we need to introduce a set $\{\mu\}$ of new degrees of freedom independent on $\boldsymbol{p}$, such that
\begin{gather*}
\hat{\varphi}\left(\mathbb{M}(\boldsymbol{p})\right)
|\boldsymbol{p},\mu\rangle
=
{\mathbb{S}(\boldsymbol{p})^\mu}_\nu
|\boldsymbol{p},\nu\rangle
\end{gather*}
The point here is that the set of matrices $\mathbb{S}$ constitute a representation of the little group on the space of indices $\{\mu\}$. In fact, because $\hat{\varphi}$ is an homomorphism we have
\begin{gather*}
\hat{\varphi}\left(\mathbb{M}_2(\boldsymbol{p})\mathbb{M}_1(\boldsymbol{p})\right)
|\boldsymbol{p},\mu\rangle
=
\hat{\varphi}\left(\mathbb{M}_2(\boldsymbol{p})\right)
\hat{\varphi}\left(\mathbb{M}_1(\boldsymbol{p})\right)
|\boldsymbol{p},\mu\rangle
=
{\mathbb{S}_2(\boldsymbol{p})^\mu}_\nu
{\mathbb{S}_1(\boldsymbol{p})^\nu}_\xi
|\boldsymbol{p},\xi\rangle
\end{gather*}
and it's not difficult to demonstrate that exists $\mathbb{S}=\mathbb{I}$ and that $\forall\,\mathbb{S}\,\exists\,\mathbb{S}^{-1}$ given the fact that the little group is indeed a group
About the physics
We deal with unitary and irreducible realizations $\hat{\varphi}$ of the $\text{O}(1,3)$ group (actually is the Poincaré group $\text{R}^4\rtimes\text{O}(1,3)$) acting on states $|\boldsymbol{v}\rangle$ of the field; this way internal products are preserved in the transformation (unitarity) and we are sure don't exist subsets of states that are more elementary than those we are dealing with (irreducibility).
Poincaré group is the semi-direct product of two groups and the representation should be irreducible for every one of the two: by the fact that $\text{R}^4$ is an abelian group, the only available irreducible representation has dimension $1$ (by Schur's II lemma), meaning that translation will act on our carrier space simply via $e^{\hat{p}^\alpha\epsilon_\alpha}$.
The generic state $|\boldsymbol{v}\rangle$ can be decomposed in states $|\boldsymbol{p},\mu\rangle$ and this imply that the little group representation on the carrier space of the indices $\{\mu\}$ is itself irreducible.
Casimir operators
Consider a Lie group, as Poincaré is. A Casimir operator $\mathbb{C}$ of a Lie group commutes with all the generators $\mathbb{J}_i$ of the group, so also with all they combinations and with the exponential of it, so with the generic group transformation
\begin{gather*}
\left[\mathbb{C},\mathbb{J}_i\right]
\doteq
\mathbb{O}\,\forall\,i
\\
\left[\mathbb{C},\theta^i\mathbb{J}_i\right]
=
\mathbb{O}
\\
\Rightarrow
\left[\mathbb{C},e^{\theta^i\mathbb{J}_i}\right]
=
\mathbb{O}
\end{gather*}
If we choose a $\text{P}$-irrep (Poincaré irreducible representation) we have from the II Schur's lemma that $\mathbb{C}=\alpha\mathbb{I}$ and $\alpha$ becomes a label attached to our irreducible representation on our carrier space
Casimir operators for Poincaré group
For Poincaré group the Casimirs are $\mathbb{P}_\alpha\mathbb{P}^\alpha$ (square of translation generators) and $\mathbb{W}_\alpha\mathbb{W}^\alpha$ (square of Pauli-Lubanski four-vectors).
The realization of the Poincaré group is irreducible and unitary: the four-momentum is by definition the generator of the $\text{R}^4$ translation group that realizes on states as
\begin{equation*}
e^{\hat{p}^\alpha \epsilon_\alpha}
\end{equation*}
Our requests imply that $\hat{p}_\alpha=\hat{p}_\alpha^\dagger$ (so it is an hermitian operator and can be applied Hilbert-Schmidt theorem) and that single eingenvalues $p^\alpha$ are one-dimensional, so that the Casimir operator $p^\alpha p_\alpha$ is actually a constant.
This means that a field state $|\boldsymbol{v}\rangle$ carrier of this particular realization of the Poincaré group, will be decomposed in momentum states with eigenvalues having all the same $p^\alpha p_\alpha=\text{const}$.
In practice
Consider a field which states $|\boldsymbol{v}\rangle$ are decomposed in $|\boldsymbol{p},\mu\rangle$ of the $p^\alpha p_\alpha >0$ class.
We can choose the reference momentum $\tilde{\boldsymbol{p}}=(mc,0,0,0)$; by solving the system
\begin{gather*}
 \begin{cases}
\mathbb{M}(\tilde{\boldsymbol{p}})\tilde{\boldsymbol{p}}=\tilde{\boldsymbol{p}}
\\
\mathbb{M}^{\text{T}}(\tilde{\boldsymbol{p}})
\eta
\mathbb{M}(\tilde{\boldsymbol{p}})
=
\eta
 \end{cases}
\end{gather*}
you recover transformations with generators that satisfy $\text{SU}(2)$ algebra (for which $\mathbb{J}^i\mathbb{J}_i$ is a Casimir with $s(s+1)$ label).
Given the fact that the set of $\mathbb{S}$ constitute and irreducible representation of the little group on the space of the indices, we can finally conclude
\begin{gather*}
 \begin{cases}
\left[\mathbb{J}_i,\mathbb{J}_j\right]
=
2\mathrm{i}\epsilon_{ijk}\mathbb{J}^k
\\
\hat{\varphi}\left(\text{SU}(2)\right)
|\tilde{\boldsymbol{p}},\mu\rangle
=
{\left(e^{\theta_i\mathbb{J}^i}\right)^\mu}_\nu
|\tilde{\boldsymbol{p}},\nu\rangle
 \end{cases}
\end{gather*}
and you can recover the form of the transformations for all the other $\boldsymbol{p}$ satisfying $p^\alpha p_\alpha=\tilde{p}^\alpha\tilde{p}_\alpha$ (because $\text{SU}(2)$ is the little group for just $\tilde{\boldsymbol{p}}=(mc,0,0,0)$).
