No, Wigner wouldn't expect to observe only $\vert \rightarrow\rangle$ because he was a smart cookie. ;-)
Let's closely examine the state of the rest of the universe as seen by Wigner at various stages of the experiment.
Initially...
Trivially, Wigner sees
$$\vert \textrm{friend}\rangle\otimes\vert \rightarrow\rangle$$
After the Friend Has Measured the Qubit Along $z$...
Now, the friend has been entangled with the spin, and the total state-vector as seen by Wigner would be
\begin{align}
\frac{1}{\sqrt{2}}\vert \textrm{friend sees}\uparrow\rangle\otimes\vert \uparrow\rangle+\frac{1}{\sqrt{2}}\vert \textrm{friend sees}\downarrow\rangle\otimes\vert \downarrow\rangle
\end{align}
After the Friend Has Sent the Signal...
\begin{align}&\frac{1}{\sqrt{2}}\vert \textrm{outcoming }\uparrow\rangle\otimes\vert \textrm{friend sends}\uparrow\rangle\otimes\vert \uparrow\rangle\\+&\frac{1}{\sqrt{2}}\vert \textrm{outcoming }\downarrow\rangle\otimes\vert \textrm{friend sends}\downarrow\rangle\otimes\vert \downarrow\rangle
\end{align}
We will call this state $\vert\psi\rangle$. Notice that Wigner does not see the state of the outcoming qubit to be $\vert \rightarrow\rangle$. He sees the outcoming qubit to be in an entangled state with the friend and the measured qubit.
When Wigner Measures!
Now, when Wigner measures the $x$ spin of the outcoming qubit, the state of the system would collapse to
Option $1$
\begin{align}
\mathbb{P}_{\rightarrow}\vert\psi\rangle =
\vert \mathrm{outcoming }\rightarrow\rangle\otimes\Big(&\langle\mathrm{outcoming }\rightarrow\vert \mathrm{outcoming }\uparrow\rangle\vert\mathrm{friend\ sent }\uparrow\rangle\otimes\vert\uparrow\rangle\\+&\langle\mathrm{outcoming }\rightarrow\vert \mathrm{outcoming }\downarrow\rangle\vert\mathrm{friend\ sent }\downarrow\rangle\otimes\vert\downarrow\rangle\Big)
\end{align}
which is a state with $+1$ eigenvalue of $x$ spin of the outcoming qubit (up to normalization). Wigner would observe this with a probability
\begin{align}
\vert\mathbb{P}_{\rightarrow}\vert\psi\rangle\vert^2 = \frac{1}{2}(\vert \langle\rightarrow\vert\uparrow\rangle\vert^2+\vert \langle\rightarrow\vert\downarrow\rangle\vert^2) = \frac{1}{2}
\end{align}
Option $2$
Similarly, the other option is that Wigner would observe the state
\begin{align}
\mathbb{P}_{\leftarrow}\vert\psi\rangle =
\vert \mathrm{outcoming }\leftarrow\rangle\otimes\Big(&\langle\mathrm{outcoming }\leftarrow\vert \mathrm{outcoming }\uparrow\rangle\vert\mathrm{friend\ sent }\uparrow\rangle\otimes\vert\uparrow\rangle\\+&\langle\mathrm{outcoming }\leftarrow\vert \mathrm{outcoming }\downarrow\rangle\vert\mathrm{friend\ sent }\downarrow\rangle\otimes\vert\downarrow\rangle\Big)
\end{align}
which is a state with $-1$ eigenvalue of $x$ spin of the outcoming qubit (up to normalization). Wigner would observe this with a probability
\begin{align}
\vert\mathbb{P}_{\leftarrow}\vert\psi\rangle\vert^2 = \frac{1}{2}(\vert \langle\leftarrow\vert\uparrow\rangle\vert^2+\vert \langle\leftarrow\vert\downarrow\rangle\vert^2) = \frac{1}{2}
\end{align}
So, indeed, Wigner would measure the $x$ spin of the particle to be left--pointing half of the times and Wigner wouldn't expect otherwise. As you can see, we derived this conclusion by doing everything w.r.t. Wigner from the beginning to the end.
Physically, what is going on is that the initial superposition $\vert \uparrow\rangle + \vert \downarrow\rangle$ of the first qubit does not get translated to the same superposition $\vert \uparrow\rangle + \vert \downarrow\rangle$ for the outcoming qubit. Rather, first, the friend and then the outcoming qubit gets entangled with the first qubit and the superposition is of the form
\begin{align}&\frac{1}{\sqrt{2}}\vert \textrm{outcoming }\uparrow\rangle\otimes\vert \textrm{friend sends}\uparrow\rangle\otimes\vert \uparrow\rangle\\+&\frac{1}{\sqrt{2}}\vert \textrm{outcoming }\downarrow\rangle\otimes\vert \textrm{friend sends}\downarrow\rangle\otimes\vert \downarrow\rangle\end{align}
This makes it so that the $\vert\uparrow\rangle$ and $\vert\downarrow\rangle$ of the outcoming qubit are not in a coherent superposition and thus, the expectation that what would reach Wigner would be $\vert \rightarrow\rangle$ is naive/wrong.
