You are essentially trying to change basis from eigenvectors of $\hat S_z$ to eigenvectors of $\hat S_x$, for example, for an arbitrary fixed S. (The other two cases are similar, and you may trivially carry them out yourself, as suggested here.) Feynman, vIII does your S = 1/2 case. The short answer is that your quantities are Wigner rotation d matrices for an angle π/2. Indeed, for arbitrary angles, these matrices are displayed next to the C-G coefficients in tabulations, such as the PDG.
First for the answers, before explanations. To find your spin 1/2 answer, you dotted
$$\langle 1/2, m_z=1/2|\leadsto (1,0), \qquad \langle 1/2, m_z=-1/2|\leadsto (0,1),$$
onto the eigenvectors of $\hat S_x$,
$$
|1/2,m_x=1/2\rangle \leadsto \begin{pmatrix} 1\\1\end{pmatrix} /\sqrt{2}, \qquad |1/2,m_x=-1/2\rangle \leadsto \begin{pmatrix} -1\\1\end{pmatrix} /\sqrt{2}.
$$
You then got the matrix of your basis change, up to a phase convention,
$$
\langle 1/2, m_z\!\!=\!\!1/2|1/2,m_x\!\!=\!\!1/2\rangle\!\! = 1/\sqrt{2}, \quad \langle 1/2, m_z\!\!=\!\!1/2|1/2,m_x\!\!=\!\!-1/2\rangle \!\!= - 1/\sqrt{2}\\
\langle 1/2, m_z\!\!=\!\!-1/2|1/2,m_x\!\!=\!\!1/2\rangle \!\!= 1/\sqrt{2}, \quad \langle 1/2, m_z\!\!=\!\!-1/2|1/2,m_x\!\!=\!\!-1/2\rangle \!\!= 1/\sqrt{2}.
$$
This orthogonal matrix associates eigenvectors of $\hat S_x$ to eigenvectors of $\hat S_z$, so $$|m_x\rangle= \sum_{m_z} |m_z\rangle \langle m_z|m_x\rangle.$$
For higher spins, you just dot the eigenvectors provided. For instance,
$$
\langle 1, m_z\!\!=\!\!1~|~1,m_x\!\!=\!\!1\rangle =1/2=\langle 1, m_z\!\!=\!\!-1|1,m_x\!\!=\!\!1\rangle \\ \langle 1, m_z\!\!=\!\!0|1,m_x\!\!=\!\!1\rangle =1/\sqrt{2} , \\
\langle 3/2, m_z\!\!=\!\!3/2 ~|~ 3/2,m_x\!\!=\!\!3/2\rangle = {1\over 2\sqrt{2}}~,\\
\langle 2, m_z\!\!=\!\!2 ~|~
2,m_x\!\!=\!\!2\rangle = {1\over 4 }~ ...
$$
But ... do you need to be finding the eigenvectors of the spin operators for arbitrary spins, by hand, all the time? Not really. Observe $$\hat S_z=e^{i{\pi/2} \hat S_y} \hat S_x e^{-i{\pi/2} \hat S_y} ,$$ that is, a right angle rotation around y rotates $\hat S_x$ to $\hat S_z$, and so diagonalizes it. It is straightforward to confirm the spin 1/2 case with standard Pauli matrix exponentiation to catch your signs,
$$
e^{i{\pi/4} \sigma_y} \sigma_x e^{-i{\pi/4} \sigma_y} = \sigma_z ~.
$$
Observe the right exponential is a rotation matrix transforming an eigenvector of $\hat S_z$ to an eigenvector of $\hat S_x$ with the same eigenvalue, just like the transformation matrix sought. So the eigenvectors and hence basis change coefficients are residing in the columns of the rotation Wigner d-matrices in the right angle limit. (It would be worth your while to confirm/check a few cases; further, in Feynman's (18.35) in that limit.)