Consider the s channel process, I use the metric convection $+---$ instead, the constraints are: Conservation of four momentum: $p_{1}^{\mu}+p_{2}^{\mu}=p^{\mu}$ and on mass shell condition $p_{i}^{2}=m_{i}^{2}$.
Calculation can be simplified using the center of mass frame, in this frame 3 momentum of colliding particles are equal in magnitude and opposite in direction: \begin{equation} \vec{p}_{1}=-\vec{p}_{2} \end{equation} then particles 4 momentum obeyed: \begin{align} p_{1}=(E_1,\vec{p}_1)\;\;\;\; p_{2}=(E_2,-\vec{p}_{1})\;\;\;\; p=(E_1+E_2,0) \end{align} Define the Lorentz invariant quantity $s$ as the square of four momentum of sum of colliding particles: \begin{equation} s=(p_1+p_2)^{2} \end{equation} As the sum of energies and rest mass energy can be measured, it's advisable to expressed the momentum in terms of these parameters, now take the scalar product of conservation of four momentum with particle $1$ momentum: \begin{equation} p\cdot p_1=p_1 \cdot p_1+p_1 \cdot p_2 \rightarrow \sqrt{s}E_{1}=m_{1}^{2}+p_{1}\cdot p_{2} \end{equation} The last term can be eliminate using the squaring of conservation of four momentum: \begin{equation} p_{1}^{2}+p_{2}^{2}+2p_{1}\cdot p_{2}=s \rightarrow p_1 \cdot p_2=\frac{s-(m_{1}^{2}+m_{2}^{2})}{2} \end{equation} Substitute this expression to obtain $E_1$ in terms of $s,m_1,m_2$: \begin{equation} E_1=\frac{1}{2\sqrt{s}}(s+m_{1}^{2}-m_{2}^{2}) \end{equation} Therefore the magnitude of 3-momentum of particle 1 can be found: \begin{equation} |\vec{p}_1|=\sqrt{E_{1}^{2}-m_{1}^{2}}=\sqrt{\frac{1}{4s}(s+m_{1}^{2}-m_{2}^{2})^{2}-m_{1}^{2}} \end{equation} A couple of steps of algebra: \begin{equation} |\vec{p}_{1}|=\sqrt{\frac{1}{4s}[s^{2}+2(m_{1}^{2}-m_{2}^{2})s+(m_{1}^{2}-m_{2}^{2})^{2}}{-4sm_{1}^{2}]} \end{equation} Finally I obtain the magnitude: \begin{equation} |\vec{p}_{1}|=\frac{1}{2\sqrt{s}}\sqrt{s^{2}-2(m_{1}^{2}+m_{2}^{2})s+(m_{1}^{2}-m_{2}^{2})^2} \end{equation} I am struggling on converting to laboratory frame to obtain the momentum of particle no.1 in this frame where the particle no.2 is initially at rest, I am not sure about how's the inverse transformation can be done.
