Yes, what you are suggesting is exactly what is happening, but that is if you have an expression which transforms like an axial vector which you can identify with the the spin of the photon. The inherent spin property of photons ($1\hbar$) and electrons ($\tfrac12\hbar$) is of course reference frame independent.
Maybe without realizing you brought forward here the issue of "what is the expression which represents the spin of the electromagnetic field", This field can not be expressed in a gauge invariant way because it contains the vector potential $A^\mu$.
The spin density in the Lorentz gauge would be:
$ {\cal C}^\mu ~~=~~ \epsilon_o\,\tfrac12\varepsilon^{\,\mu\nu\alpha\beta} F_{\alpha\beta}A_\nu ~~=~~ \epsilon_o\,\varepsilon^{\,\mu\alpha\beta\gamma} A_\alpha\partial_\beta A_\gamma
$
Which (in vacuum) is equal to.
$ {\cal C}^\mu ~~=~~
\left(
\begin{array}{c c c c}
~ 0 &-\tfrac1c\,\mathsf{H}_x &-\tfrac1c\,\mathsf{H}_y &-\tfrac1c\,\mathsf{H}_z \\
\tfrac1c\,\mathsf{H}_x & ~~~ 0 & \ \ ~~\mathsf{D}_z & ~-\mathsf{D}_y \\
\tfrac1c\,\mathsf{H}_y & ~-\mathsf{D}_z & ~~~ 0 & \ \ ~~\mathsf{D}_x \\
\tfrac1c\,\mathsf{H}_z & \ \ ~~\mathsf{D}_y & ~-\mathsf{D}_x & ~~~ 0
\end{array}
\right) \left(
\begin{array}{c}
\ \ A_0 \\
-A_x \\
-A_y \\
-A_z
\end{array}
\right)
$
From this expression one can already see that it transforms like an axial vector. If you go through the trouble of calculating the $A^\mu$ field of a circulating charge using Liénard Wiechert (like i did here) then you get indeed the required $1\hbar$ ratio with the momentum density for circular polarized photons and $0\hbar$ for linear polarized photons.
The latter expression is equivalent to the electron's spin density found via the Gordon decomposition of the axial Dirac current of the electron. In this case the matrix is given by the Magnetization Polarization tensor of the Dirac field while the column vector is given by the dynamic momentum of the electron. (The phase change rates minus the phase induced by the $A^\mu$ field, $\partial_\mu-ieA_\mu$).
