Suppose there are two $\gamma$ rays with frequencies $\nu_1$ and $\nu_2$ moving in opposite directions according to a reference frame $S$. I want to find the velocity of the center of mass of this system.
Since photons do not have mass, the center of mass is the frame in which the sums of momenta vanishes.
Let $S'$ be this reference frame. The total momentum in $S'$ is given by:
$$ p'= p_{1}'+p_{2}' = \frac{h}{c}(\nu_{1}'-\nu_{2}')=0 $$
Which implies $\nu_{1}'=\nu_{2}'$.
There frequencies in $S'$ are given by: $$ \nu'= \left(\frac{1+\beta}{1-\beta}\right)^{1/2}\nu $$
Therefore, the condition $\nu_{1}'=\nu_{2}'$ gives $\nu_{1}=\nu_{2}$. But $\nu_{1} \neq \nu_{2}$ because the photons can have different frequencies in $S$.
What has gone wrong in the reasoning?