Confusion about rotations of quantum states: $SO(3)$ versus $SU(2)$ I'm trying to understand the relationship between rotations in "real space" and in quantum state space. Let me explain with this example: 
Suppose I have a spin-1/2 particle, lets say an electron, with spin measured in the $z+$ direction. If I rotate this electron by an angle of $\pi$ to get the spin in the $z-$ direction, the quantum state "rotates" half the angle $(\pi/2)$ because of the orthogonality of the states $|z+\rangle$ and $|z-\rangle$. I think this isn't very rigorous, but is this way of seeing it correct?
I searched for how to derive this result, and started to learn about representations. I read about $SO(3)$ and $SU(2)$ and their relationship, but it's still unclear to me. I found this action of $SU(2)$ on spinors:
$$ \Omega(\theta, \hat n) = e^{-i \frac{\theta}{2}(\hat n\cdot \vec\sigma)}, $$
 where $\hat n$ is a unitary $3D$ vector, $\vec\sigma = (\sigma_1,\sigma_2,\sigma_3)$ is the Pauli vector and $\sigma_i$ are the Pauli matrices. I see the factor of $\frac{1}{2}$ on the rotation angle $\theta$, but where does it come from? 
I saw $[\sigma_i,\sigma_j] = 2i\epsilon_{ijk}\sigma_k $, and making $ X_j = -\frac{i}{2}\sigma_j $ the commutator becomes $[X_i,X_j] = \epsilon_{ijk}X_k$, which is the commutator of the $\mathfrak{so}(3)$ Lie algebra, isn't it? So when I compute the exponential $$e^{\theta(\hat n\cdot X)} = e^{-i \frac{\theta}{2}(\hat n\cdot \vec\sigma)}.$$ I get my result, and it seems like it's a rotation, but I read that it isn't an $SO(3)$ representation. So where do rotations appear?
However, my central question is: How can I demonstrate that a rotation on "our world" generates a rotation of quantum states, and how do I use that to show the formula for rotations on quantum states? And how I do it for higher spin values? I'm really new on this topic, and it was hard to formulate this question, so feel free to ask me for a better explanation or to clear any misconception.
 A: This is a very good question. There are several ways to answer your question. I remember understanding all the group theory and yet feeling I didn't understand the physics when I first encountered the problem so I will try to explain things physically. 
First one should ask how one would you know something has angular momentum and measure it? I am not talking about spin only but also every day objects. 
You would couple it to something else that can pick up the angular momentum. Like a baseball hitting bat. 
In case of electrons it comes from the spin-orbit coupling or emission of photons that carry spin. However, we have a much more accessible example because electrons carry charge. The charge helps us as the angular momentum then manifests itself as magnetic moment which couples to a magnetic field and we can use the Stern-Gerlach set up to influence the trajectories of particles with different angular momentum (as long as they are charged under the usual U(1) EM charge).
So basically the hamiltonian has a coupling
$$
\Delta H = g ~ \hat n. \vec J
$$
where in the Stern Gerlach case the $\hat n$ corresponds to the magnetic field direction. Invariance under infinitestimal rotations means the $J$ have to satisfy the algebra (you should try to prove this or ask a separate question)
$$
[J_i,J_j]=\epsilon_{ijk} J_k 
$$
where summation convention on repeated indices is used. The smallest dimension operators that satisfy it are $2 \times 2$ (a result otherwise known as the smallest spin being $\frac{1}{2}$) and these operators are spanned by the basis $\{\frac{1}{2} \sigma_x,\dots \}$.
At this point note a mathematical fact that I have never tried to prove, for half-intergral angular momentum operators, exponentiation does not cover the rotation group $SO(3)$ but instead covers $SU(2)$. These are locally isomorphic and that is all that is required from the coupling in the Hamiltonian. In other words representations of $SU(2)$ can couple to classical instruments that measure angular momentum (doesn't mean they have to for instance SU(2) subset of color SU(3) does not).
So, we conclude that for spin $\frac{1}{2}$ the most general angular momentum operator is 
$$
\mathcal O = \frac{1}{2} \hat n . \vec \sigma
$$
where $\hat n$ is a unit vector in some direction.
Now we can do a lot of mathematics and group/representation theory to say this transforms in the adjoint representation or we can see the physics. Suppose we perform a rotation of angle $\theta$ around the axis given by unit vector $\hat m$. What do we expect? We'd expect to get a new opertor that corresponds to rotating $\hat n$ by $\theta$ around $\hat m$. Such a vector is
$$
\tilde {\hat n}= (\hat n . \hat m) \hat m + ( \hat n - (\hat n . \hat m) \hat m) \cos \theta + (\hat m \times \hat n) \sin \theta
$$
and the rotated operator is
$$
\tilde {\mathcal O} = \frac{1}{2} \tilde {\hat n}. \vec \sigma
$$
Now the claim is that the operator $\mathcal O$ transforms in the adjoint rep and that means we should get 
$$
\tilde {\mathcal O}=  \frac{1}{2} e^{-i\frac{\theta}{2} \hat m.\vec \sigma} ~\hat n. \vec \sigma ~e^{i\frac{\theta}{2} \hat m.\vec \sigma}
$$
It indeed does and for completeness I will derive it here. We will need
$$
(\vec a. \vec \sigma)(\vec b. \vec \sigma) = (\vec a.\vec b) +i (\vec a \times \vec b).\vec \sigma
$$
We see that
$$
\begin{align}
& \frac{1}{2} e^{-i\frac{\theta}{2} \hat m.\vec \sigma} ~\hat n. \vec \sigma ~e^{i\frac{\theta}{2} \hat m.\vec \sigma} \\
&=[ \cos(\frac{\theta}{2}) - i (\hat m.\vec \sigma) \sin(\frac{\theta}{2})] [ \hat n.\vec \sigma] [ \cos(\frac{\theta}{2}) + i (\hat m.\vec \sigma) \sin(\frac{\theta}{2})] \\
&=[ \cos(\frac{\theta}{2}) - i (\hat m.\vec \sigma) \sin(\frac{\theta}{2})] [ (\hat n.\vec \sigma) \cos(\frac{\theta}{2}) +i ( \hat n.\hat m + i (\hat n \times \hat m).\vec \sigma) \sin(\frac{\theta}{2})] \\
&=(\hat n. \vec \sigma) \cos^2 ( \frac{\theta}{2}) + 2 ( \hat m \times \hat n) \sin(\frac{\theta}{2}) \cos (\frac{\theta}{2}) +(\hat n.\hat m) \hat m .\vec \sigma \sin^2 (\frac{\theta}{2}) \\
&~~~-( \hat n - (\hat n.\hat m) \hat m).\vec \sigma \sin^2(\frac{\theta}{2}) \\
&=\tilde {\hat n} .\vec  \sigma
\end{align}
$$
So we see that indeed the operator transforms in the adjoint representation.
Now the expectation value for a state $|\psi \rangle$ is
$$
\langle \psi | \mathcal O | \psi \rangle
$$
and therefore invariance under rotations means that
$$
\begin{align}
| \tilde \psi \rangle &= e^{-i\frac{\theta}{2} \hat m.\vec \sigma} |\psi \rangle \\
&= [\cos(\frac{\theta}{2}) - i (\hat m.\vec \sigma) \sin (\frac{\theta}{2})]|\psi \rangle
\end{align}
$$
which shows that under a rotation by $\pi$ we get $| \tilde \psi \rangle = -i (\hat m. \vec \sigma) | \psi \rangle$ as the OP asked.
The generalization to higher spins is not straightforward as we do not have properties like $J_x^2=1$ and one must use Baker–Campbell–Hausdorff formula. Nevertheless, once the above idea is clear one can use group theory results to see that for spin-1 for instance, we will have 3 $3 \times 3$ matrices 
$$
\begin{align}
  J_x &= \frac{1}{\sqrt{2}}
    \begin{pmatrix}
      0 &1 &0\\
      1 &0 &1\\
      0 &1 &0
    \end{pmatrix} \\
  J_y &= \frac{1}{\sqrt{2}}
    \begin{pmatrix}
      0 &-i &0\\
      i &0  &-i\\
      0 &i  &0
    \end{pmatrix} \\
  J_z &=
    \begin{pmatrix}
      1 &0 &0\\
      0 &0 &0\\
      0 &0 &-1
   \end{pmatrix}
\end{align}
$$
and again the most general operator will be
$$
\mathcal O_3 = \hat n.\vec J
$$ 
Now since the operator is made of the generators of SO(3) we know that it will transform under the adjoint rep and, while it will considerably more work than above to prove it is so, the meaning of such transformation is that
$$
\tilde {\mathcal O_3} = \tilde{\hat n}.\vec J
$$
and then the same argument as above will give us that the state transforms in the fundamental representation as
$$
| \tilde {\psi_3} \rangle = e^{-i \theta ~ \hat n. \vec J} |\psi_3 \rangle
$$
Alternately, one makes the spin-1 particle as the triplet state of two spin-$\frac{1}{2}$ particles and works from there.
A: 
How can I demonstrate that a rotation on "our world" generates a rotation of quantum states, and how do I use that to show the formula for rotations on quantum states?

As with anything in physics, you make postulates and try them out. However, the postulates you need to make are very, very mild and obvious and then these allow you to use Wigner's Theorem, which is a powerful beast that really nails things down precisely for the question you ask above.
When we rotate our quantum object or our co-ordinate system, we make the very obvious postulate that, whatever transformation this action brings about on the quantum state space, this transformation must preserve the inner products in the quantum state space so that the state stays properly normalised.
From these postulates alone,  i.e. one does not even have to assume linearity, Wigner proved that the when our quantum object / co-ordinate system undergoes  a number of consecutive "symmetries" (i.e. Lorentz transformations, which of course include rotations), the corresponding quantum state space transformations must "compose compatibly with the symmetry composition" (the jargon phrase one hears in this context). This is more precisely stated in symbols. Let $\sigma:\mathrm{SO}(1,\,3)\to T$ be the mapping between the "symmetries" (Lorentz transformations, including rotations in $\mathrm{SO}(1,\,3$) and the set of quantum state space transformations $T$. Then we have:

Wigner's theorem
Given the postulates above (see elsewhere for a more precise statement), for any two $\gamma,\,\zeta\in\mathrm{SO}(1,\,3)$, the corresponding quantum state space transformations $\sigma(\gamma)$ and $\sigma(\zeta)$:


*

*Act linearly or antilinearly on the quantum state space (and are obviously unitary / antiunitary, since they have to conserve inner products); and

*$\sigma(\gamma\,\zeta) = \pm \sigma(\gamma)\,\sigma(\zeta)$



i.e. the mapping $\sigma$ is a so-called projective homomorphism.
The $\pm$ sign in the equation above is both inconsequential and weighty at the same time! It's inconsequential in the sense that, since quantum states are actually rays in the quantum state space, the sign (like any global phase) has no effect on the transformation's physical action on the state space. But the fact that there is this sign there means that:


*

*Not only are we dealing with representations of the group $\mathrm{SO}(1,\,3)$ (or $\mathrm{SO}(3)$ in the context of your question) (which is the "+" sign case); BUT

*We could be dealing with the representation of a Topological Covering Space of the group $\mathrm{SO}(1,\,3)$ or $\mathrm{SO}(3)$ (in which case our $\sigma$ can add the $-$ phase).


The idea of topological covering space is key here. For $\mathrm{SO}(1,\,3)$ and $\mathrm{SO}(3)$ there are only two possible covers each: the groups themselves and their universal covers $\mathrm{SL}(2,\,\mathbb{C})$ and $\mathrm{SU}(2)$, respectively. There aren't any more because both $\mathrm{SL}(2,\,\mathbb{C})$ and $\mathrm{SU}(2)$ are simply connected, and a  simply connected topological space admits no non trivial covering spaces. There is a readable proof of this in the old book by W. S. Massey, "Algebraic Topology: An Introduction".
So, the crucial relationship is that $\mathrm{SU}(2)$ is the double (and universal, therefore the only nontrivial) cover of $\mathrm{SO}(3)$.
Now both Quantum Mechanic's answer and Borun Chowdhury's more physics answer give you quite a bit of details of the calculations in $SU(2)$ and $SO(3)$ and of their representations but if you want to try to visualize their relationship, then perhaps have a look at the latter half of this answer of mine where I try to illustrate the standard construction of the topological universal covering group applied to $SO(3)$ so that you can visualize $SU(2)$ as being made of two copies of $SO(3)$.
A: *

*On one hand, if $\vec{\alpha}\in \mathbb{R}^3$ denotes a 3D rotation vector$^1$, then the corresponding rotation matrix
$$R(\vec{\alpha})~=~\exp(i\vec{\alpha}\cdot \vec{L})~\in~ SO(3)~ \subseteq ~{\rm Mat}_{3\times 3}(\mathbb{R}). \tag{1}$$
Here $iL_j\in so(3)  \subseteq  {\rm Mat}_{3\times 3}(\mathbb{R})$ are three $so(3)$ Lie algebra generators, defined as
$$\begin{align} i(L_j)_{k\ell} ~=~& \epsilon_{jk\ell} ,\cr j,k,\ell~\in~& \{1,2,3\}, \cr \epsilon_{123}~=~&1,\tag{1'}\end{align}$$
and fulfilling the $so(3)$ Lie bracket relations
$$\begin{align}   [L_j, L_k] ~=~& i\sum_{\ell=1}^3\epsilon_{jk\ell} L_{\ell}, \cr j,k,\ell~\in~& \{1,2,3\}.\end{align} \tag{1"}$$


*On the other hand, the corresponding $SU(2)$ matrix is$^2$
$$ X(\vec{\alpha})~=~\exp(\frac{i}{2}\vec{\alpha}\cdot \vec{\sigma})~\in~ SU(2)~ \subseteq ~{\rm Mat}_{2\times 2}(\mathbb{C}), \tag{2}$$
where $\sigma_{\ell}$ are the Pauli matrices.


*The two matrix representations (1) & (2) are related via
$$\begin{align} X(\vec{\alpha}) \sigma_k X(\vec{\alpha})^{-1}~=~& \sum_{j=1}^3\sigma_j R(\vec{\alpha})^j{}_k, \cr 
  R(\vec{\alpha})^j{}_k~=~&\frac{1}{2}{\rm Tr}(\sigma^jX(\vec{\alpha}) \sigma_k X(\vec{\alpha})^{-1}),
\cr j,k~\in~& \{1,2,3\}. \end{align}\tag{3}$$


*The relation (3) exposes the fact that the adjoint representation
$$ {\rm Ad}:~ SU(2) \quad\to\quad SO(su(2))~\cong~ SO(3),\tag{4} $$
given by
$$ \begin{align}
X\quad \mapsto&\quad{\rm Ad}(X)\sigma~:= ~X\sigma X^{-1},
\cr X~\in~& SU(2),\cr
\sigma~\in~& su(2)\cr
~:=~&\{ \sigma\in {\rm Mat}_{2\times 2}(\mathbb{C})\mid \sigma^{\dagger}=\sigma~\wedge~{\rm tr}(\sigma)=0\}\cr
~=~&{\rm span}_{\mathbb{R}}\{\sigma_1,\sigma_2,\sigma_3\},\cr
&||\sigma||^2  ~=~\det(\sigma),\cr
&{\rm Ad}(\pm {\bf 1}_{2 \times 2})~=~{\bf 1}_{su(2)}, 
 \end{align}\tag{4'} $$
is a surjective 2:1 Lie group homomorphism between
$SU(2)$ and $SO(3)$, i.e.
$$ SU(2)/\mathbb{Z}_2~\cong~SO(3). \tag{4"}$$


*See also this related Phys.SE post, which explains the half-angle appearance in eq. (2).
References:

*

*G. 't Hooft, Introduction to Lie Groups in Physics, lecture notes, chapters 3 & 6. The pdf file is available here.

--
$^1$ The direction of a rotation vector $\vec{\alpha}\in \mathbb{R}^3$ is parallel to the rotation axis, while the length $|\vec{\alpha}|$ denotes the counter-clockwise rotation angle (as seen from the tip of the rotation vector).
$^2$ The notation and conventions in this Phys.SE answer follows Ref. 1.
