I am confused about the case where mass is not conserved in a collision (not due to relativistic factors). The center of momentum (CoM) frame is not the same before and after the collision.
Let's call frame 1 the initial CoM frame, and frame 2 the final CoM frame. So on the one end in frame 2: $p^{(2)}_{final}=0$ because it is the CoM frame. But on the other end in frame 1: $p^{(1)}_{initial}=0$, because it is also the CoM frame. As I understand it, there cannot be another frame where $p_{initial}=0$ apart from frame 1, therefore $p^{(2)}_{initial}\neq0$. So 4 momentum is not conserved.
Actually the Wiki page on CoM frame suggests there are several CoM frames, but I am assuming they do not differ by velocity but rather by point of origin?
In the simpler, non relativistic case, I found that by calculating $p_{initial}$ in the 2nd frame:
$$p^{(2)}_{initial} = p^{(Lab)}_{initial} \cdot \text{const} \neq 0$$
EDIT:
Here is the non-relativistic calculation (relativistic does not help). The process (in lab frame) is: $(M,\bar{\omega}) + (m,\bar{u}) \rightarrow (M+\delta,\bar{v}) + (m,\bar{u'})$. The velocity of the center of momentum in frame 2 is $\bar{s}^*$.
$$M^*=M+\delta \qquad \mu = \frac{M}{m} \qquad \mu^*=\frac{M^*}{m} \qquad \bar{s}^* = \frac{\mu^*\bar{v}+\bar{u'}}{1+\mu^*}$$
$$p^{(2)}_{initial}=M (\bar{w}-\bar{s^*}) + m (\bar{u}-\bar{s^*})=\left\{ M \bar{w} + m \bar{u} \right\} - \left\{ M \bar{s^*} + m \bar{s^*} \right\} $$ Using conservation of momentum in lab frame: $$= \left\{ M^* \bar{v} + m \bar{u}' \right\} - \left\{ M \bar{s^*} + m \bar{s^*} \right\}$$ $$= m \left( \left\{ \mu^* \bar{v} + \bar{u}' \right\} - \bar{s^*} \left\{ \mu + 1 \right\} \right)$$ $$= m \left( \left\{ \mu^* \bar{v} + \bar{u}' \right\} - \frac{\mu^*\bar{v}+\bar{u'}}{1+\mu^*} \left\{ \mu + 1 \right\} \right)$$ $$= m \left\{ \mu^* \bar{v} + \bar{u}' \right\} \left( 1 - \frac{1}{1+\mu^*} \left\{ \mu + 1 \right\} \right)$$ $$= \left\{ M^* \bar{v} + m \bar{u}' \right\} \frac{\delta }{\delta +m+M}$$ $$= p^{(2)}_{initial} = p^{(lab)}_{initial} \frac{\delta }{\delta +m+M} \neq 0$$
But hey, you might think $\bar{s^*}$ isn't the right velocity, well lets see; Here is the (final) momentum in the center of momentum frame:
$$p^{(2)}_{final} = M^* (\bar{v}-\bar{s^*}) + m (\bar{u'}-\bar{s^*}) $$ $$= M^* \left(\bar{v}-\frac{\mu^*\bar{v}+\bar{u'}}{1+\mu^*}\right) + m \left(\bar{u'}-\frac{\mu^*\bar{v}+\bar{u'}}{1+\mu^*}\right)$$ $$= \left(M^* \bar{v}+m \bar{u'}\right) - (m + M^*) \left(\frac{\mu^*\bar{v}+\bar{u'}}{1+\mu^*}\right)$$ $$= \left(M^* \bar{v}+m \bar{u'}\right) - (m + M^*) \left(\frac{M^*\bar{v}+m \bar{u'}}{m + M^*}\right)$$ $$= \left(M^* \bar{v}+m \bar{u'}\right) - \left(M^*\bar{v}+m \bar{u'}\right) = 0$$
What am I not taking into consideration? What is the initial state of frame 2, $p^{(2)}_{initial}$?