I can't check your error , but here is how you can obtain the angular velocity vector

the angular velocity vector is:

$$ \vec \omega=\frac {\vec R\times \vec V}{\vec R\cdot\vec R}\tag 1 $$

with :
$$\vec R=\left[ \begin {array}{c} x\\y\\0
\end {array} \right] 
\quad,\vec V=\left[ \begin {array}{c} v_{{x}}\\ v_{{y}}
\\ 0\end {array} \right]\quad\Rightarrow \quad
\vec\omega_{xy}= \left[ \begin {array}{c} 0\\0\\ 
{\frac {-yv_{{x}}+xv_{{y}}}{{x}^{2}+{y}^{2}}}\end {array} \right] 
$$

and with 

$$ \vec R'=S\,\vec R\quad,\vec V'=S\,\vec V\quad,
S= \left[ \begin {array}{ccc} \cos \left( \phi \right) &\sin \left( \phi
 \right) &0\\ -\sin \left( \phi \right) &\cos
 \left( \phi \right) &0\\ 0&0&1\end {array} \right]$$

you obtain from equation (1)

$$ \vec \omega'=\frac {\left(\vec{S\,R}\right) \times \left(\vec{S\,V}\right)}{\left(\vec{S\,R}\right)^T\,\left(\vec{S R}\right)}= \frac {\vec R\times \vec V}{\vec R\cdot\vec R}=\vec \omega$$

if you have one rotation angle ,the transformation matrix $~\mathbf S~$ has no effect on the results 

------------------------------------------------

with 
$$\vec R=\begin{bmatrix}
  0 \\
  y \\
  z\\
\end{bmatrix}\quad,\vec V=\begin{bmatrix}
  0 \\
  v_y \\
  v_z\\
\end{bmatrix}\quad,
S= \left[ \begin {array}{ccc} 1&0&0\\  0&\cos \left( 
\phi \right) &\sin \left( \phi \right) \\  0&-\sin
 \left( \phi \right) &\cos \left( \phi \right) \end {array} \right] \\
\vec \omega_{yz}= \left[ \begin {array}{c} {\frac {-zv_{{y}}+yv_{{z}}}{{y}^{2}+{z}^{2}}
}\\ 0\\ 0\end {array} \right] 
$$