Understanding the Bloch sphere It is usually said that the points on the surface of the Bloch sphere represent the pure states of a single 2-level quantum system. A pure state being of the form:
$$
|\psi\rangle = a |0\rangle+b |1\rangle
$$
And typically the north and south poles of this sphere correspond to the $|0\rangle$ and $|1\rangle$ states. Image: ("Bloch Sphere" by Glosser.ca - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/File:Bloch_Sphere.svg#/media/File:Bloch_Sphere.svg)



*

*But isn't this very confusing? If the north and south poles are chosen, then both states are on the same line and not orthogonal anymore, so how can one choose an arbitrary point $p$ on the surface of the sphere and possibly decompose it in terms of $0,1$ states in order to find $a$ and $b$? Does this mean that one shouldn't regard the Bloch sphere as a valid basis for our system and that it's just a visualization aid?

*I have seen decompositions in terms of the internal angles of the sphere, in the form of: $a=\cos{\theta/2}$ and $b=e^{i\phi}\sin{\theta/2}$ with $\theta$ the polar angle and $\phi$ the azimuthal angle. But I am clueless as to how these are obtained when $0,1$ states are on the same line.
 A: The Bloch sphere is beautifully minimalist.
Conventionally, a qubit has four real parameters; $$|\psi\rangle=a e^{i\chi} |0\rangle + b e^{i\varphi} |1\rangle.$$ However, some quick insight reveals that the a-vs-b tradeoff only has one degree of freedom due to the normalization a2 + b2 = 1, and some more careful insight reveals that, in the way we construct expectation values in QM, you cannot observe χ or φ themselves but only the difference χ – φ, which is 2π-periodic. (This is covered further in the comments below but briefly: QM only predicts averages $\langle \psi|\hat A|\psi\rangle$ and shifting the overall phase of a wave function by some $|\psi\rangle\mapsto e^{i\theta}|\psi\rangle$ therefore cancels itself out in every prediction.)
So if you think at the most abstract about what you need, you just draw a line from 0 to 1 representing the a-vs-b tradeoff: how much is this in one of these two states? Then you draw circles around it: how much is the phase difference? What stops it from being a cylinder is that the phase difference ceases to matter when a = 1 or b = 1, hence the circles must shrink down to points. And voila, you have something which is topologically equivalent to a sphere. The sphere contains all of the information you need for experiments, and nothing else.
It’s also physical, a real sphere in 3D space.
This is the more shocking fact. Given only the simple picture above, you could be forgiven for thinking that this was all harmless mathematics: no! In fact the quintessential qubit is a spin-½ system, with the Pauli matrices indicating the way that the system is spinning around the x, y, or z axes. This is a system where we identify $$|0\rangle\leftrightarrow|\uparrow\rangle, \\
|1\rangle\leftrightarrow|\downarrow\rangle,$$ and the phase difference comes in by choosing the +x-axis via $$|{+x}\rangle = \sqrt{\frac 12} |0\rangle + \sqrt{\frac 12} |1\rangle.$$
The orthogonal directions of space are not Hilbert-orthogonal in the QM treatment, because that’s just not how the physics of this system works. Hilbert-orthogonal states are incommensurate: if you’re in this state, you’re definitely not in that one. But this system has a spin with a definite total magnitude of $\sqrt{\langle L^2 \rangle} = \sqrt{3/4} \hbar$, but only $\hbar/2$ of it points in the direction that it is “most pointed along,” meaning that it must be distributed on some sort of “ring” around that direction. Accordingly, when you measure that it’s in the +z-direction it turns out that it’s also sort-of half in the +x, half in the –x direction. (Here “sort-of” means: it is, if you follow up with an x-measurement, which will “collapse” the system to point → or ← with angular momentum $\hbar/2$ and then it will be in the corresponding “rings” around the x-axis.)
Spherical coordinates from complex numbers
So let’s ask “which direction is the general spin-½ $|\psi\rangle$ above, most spinning in?” This requires constructing an observable.
To give an example observable, if the +z-direction is most-spun-in by a state $|\uparrow\rangle$ then the observable for $z$-spin is the Pauli matrix $$\sigma_z = |\uparrow\rangle\langle\uparrow| - |\downarrow\rangle\langle\downarrow|=\begin{bmatrix}1&0\\0&-1\end{bmatrix},$$which is +1 in the state it's in, -1 in the Hilbert-perpendicular state $\langle \downarrow | \uparrow \rangle = 0.$
Similarly if you look at $$\sigma_x = |\uparrow\rangle \langle \downarrow | + |\downarrow \rangle\langle \uparrow |=\begin{bmatrix}0&1\\1&0\end{bmatrix},$$ you will see that the $|{+x}\rangle$ state defined above is an eigenvector with eigenvalue +1 and similarly there should be a $|{-x}\rangle \propto |\uparrow\rangle - |\downarrow\rangle$ satisfying $\langle {+x}|{-x}\rangle = 0,$ and you can recover $\sigma_x = |{+x}\rangle\langle{+x}| - |{-x}\rangle\langle{-x}|.$
So, let’s now do it generally. The state orthogonal to $|\psi\rangle = \alpha |0\rangle  + \beta |1\rangle$ is not too hard to calculate as $|\bar \psi\rangle = \beta^*|0\rangle - \alpha^* |1\rangle,$ so the observable which is +1 in that state or -1 in the opposite state is:$$
\begin{align}
|\psi\rangle\langle\psi| - |\bar\psi\rangle\langle\bar\psi| &= \begin{bmatrix}\alpha\\\beta\end{bmatrix}\begin{bmatrix}\alpha^*&\beta^*\end{bmatrix} - \begin{bmatrix}\beta^*\\-\alpha^*\end{bmatrix} \begin{bmatrix}\beta & -\alpha\end{bmatrix}\\
&=\begin{bmatrix}|\alpha|^2 - |\beta|^2 & 2 \alpha\beta^*\\
2\alpha^*\beta & |\beta|^2 - |\alpha|^2\end{bmatrix}
\end{align}$$Writing this as $v_i \sigma_i$ where the $\sigma_i$ are the Pauli matrices we get:$$v_z = |\alpha|^2 - |\beta|^2,\\
v_x + i v_y = 2 \alpha^* \beta.$$
Now here's the magic, let's allow the Bloch prescription of writing $$\alpha=\cos\left(\frac\theta2\right),~~\beta=\sin\left(\frac\theta2\right)e^{i\varphi},$$ we find out that these are:$$\begin{align} v_z &= \cos^2(\theta/2) - \sin^2(\theta/2) &=&~ \cos \theta,\\
v_x &= 2 \cos(\theta/2)\sin(\theta/2) ~\cos(\phi) &=&~ \sin \theta~\cos\phi, \\
v_y &= 2 \cos(\theta/2)\sin(\theta/2) ~\sin(\phi) &=&~ \sin \theta~\sin\phi.
\end{align}$$So the Bloch prescription uses a $(\theta, \phi)$ which are simply the spherical coordinates of the point on the sphere which such a $|\psi\rangle$ is “most spinning in the direction of.”
So instead of being a purely theoretical visualization, we can say that the spin-½ system, the prototypical qubit, actually spins in the direction given by the Bloch sphere coordinates! (At least, insofar as a spin-up system spins up.) It is ruthlessly physical: you want to wave it away into a mathematical corner and it says, “no, for real systems I’m pointed in this direction in real 3D space and you have to pay attention to me.”
How these answer your questions.

*

*Yes, N and S are spatially parallel but in the Hilbert space they are orthogonal. This Hilbert-orthogonality means that a system cannot be both spin-up and spin-down. Conversely the lack of Hilbert-orthogonality between, say, the z and x directions means that when you measure the z-spin you can still have nonzero measurements of the spin in the x-direction, which is a key feature of such systems. It is indeed a little confusing to have two different notions of “orthogonal,” one for physical space and one for the Hilbert space, but it comes from having two different spaces that you’re looking at.


*One way to see why the angles are physically very useful is given above. But as mentioned in the first section, you can also view it as a purely mathematical exercise of trying to describe the configuration space with a sphere: then you naturally have the polar angle as the phase difference, which is $2\pi$-periodic, so that is a naturally ‘azimuthal’ coordinate; therefore the way that the coordinate lies along 0/1 should be a ‘polar’ coordinate with 0 mapping to $|0\rangle$ and π mapping to $|1\rangle$. The obvious way to do this is with $\cos(\theta/2)$ mapping from 1 to 0 along this range, as the amplitude for the $|0\rangle$ state; the fact that $\cos^2 + \sin^2 = 1$ means that the $|1\rangle$ state must pick up a $\sin(\theta/2)$ amplitude to match it.
A: A mere extended comment streamlining the fine answer of @Timaeus  to a more memorable form.
The state vector
$$ |\psi\rangle=   \begin{pmatrix}
      \cos \theta/2          \\
         e^{i\phi} \sin \theta/2
    \end{pmatrix}$$
defines a pure state density matrix through its projection operator,
$$\bbox[yellow]{
|\psi\rangle \langle \psi | =  \begin{pmatrix}
      \cos^2 \theta/2             &  \sin \theta/2 ~ \cos\theta/2 ~e^{-i\phi} \\
         \sin \theta/2 ~ \cos\theta/2 ~e^{i\phi} &  \sin^2 \theta/2
    \end{pmatrix}=\rho  }~.
$$ 
Note the manifest invariance under over-all rephasing of $|\psi\rangle$. 
The general principles' expression of this idempotent hermitean density matrix is also, evidently,   $$
\rho=\frac{1}{2}(1\!\! 1 +  \hat n  \cdot \vec \sigma) ,
 $$
with  $\hat n = (\sin \theta \cos \phi, \; \sin \theta  \sin \phi, \; \cos \theta)^T. $
That is, the $\hat z$ axis rotates to the $\hat n$ axis by full (adjoint) rotation angles, specifying a half-angle (spinor, fundamental) operator expression.
A: You can associate points on the surface of a unit sphere with pure spin states in the following simple way.
A point of the sphere $(n_x,n_y,n_z)$ is associated with an eigenvector of the operator $n_x\sigma_x+n_y\sigma_y+n_z\sigma_z$ with a positive eigenvalue and vice versa. This includes all spin 1/2 single particle spin states.
And this is not random or visualization or mathematics. If you have a Stern-Gerlach device with a magnetic field inhomogeneity pointing in the direction $(n_x,n_y,n_z)$ then it will consistently deflect that beam in a particular direction when it has that state that is eigen to $n_x\sigma_x+n_y\sigma_y+n_z\sigma_z.$

But isn't this very confusing? If the north and south poles are chosen, then both states are on the same line and not orthogonal anymore,

It isn't confusing in the slightest. The geometry is related to the orientation of the physical device in the lab to which your state gives reliable results. The oppositely oriented device gives reliable results too. This is common for orthogonal states that teonorthgonal states can be eigen to the same operator.
So different points of the Bloch sphere identify different orientations that give the "up" result for different states. Do not confuse the orientation of the measurement device in 3d space with the geometry of the states in spin space.

so how can one choose an arbitrary point $p$ on the surface of the sphere and possibly decompose it in terms of $0,1$ states in order to find $a$ and $b$? 

Its the other way around. How did you decide to call some state 0 and another 1? You picked a random orientation and called it z and oriented your device to have the magnetic field inhomogeneity point that way. That gave you an up and a down. 
But now we can specify any spin state. You same you have an arbitrary point $(n_x,n_y,n_z)$ then find the eigen vector of $n_x\sigma_x+n_y\sigma_y+n_z\sigma_z.$ with positive eigen value. Call it $\left|s\right\rangle,$ then $$\left|s\right\rangle=\langle 0\left|s\right\rangle\left|0\right\rangle+\langle 1\left|s\right\rangle\left|1\right\rangle$$ so there is your $a$ and $b$ except you don't know the overall phase and magnitude but a single particle spin state doesn't have one of those.

Does this mean that one shouldn't regard the Bloch sphere as a valid basis for our system and that it's just a visualization aid?

No, it means you shouldn't confuse then geometry in the lab with the geometry of the Hilbert space. Physics is an experimental science so they are most definitely related but they are not the same.
If you want to project a vector onto an eigenspaces you don't project the labels onto each other. You can have a spin state and another spin state and when you put one through a Stern-Gerlach device oriented for the other then the spatial degrees of freedom split and separate into one that is up in that direction and one that is spatially down from that direction and the spin state literally changes to point up in the beam that spatially went up and to point down in the beam that went down. So the one particle's spin has become entangled with its own position.
The size of the Hilbert Projection tells you the size of the spatial parts that got deflected and split. But you also don't literally need to remember rules like that. If you write down the Schrödinger equation for the Stern-Gerlach device the beam splits and separates into the correct size parts and the spins align into the two polarizations and it happens without you telling it to do that.
So then the spin state is clear. It is telling you the direction it will reliably go if you give it a chance. And if you put it in a differently oriented Stern-Gerlach it will be forced to go in one of the two directions allowed by that orientation and it will split and go in both. To get the sizes of each part you can evolve the Schrödinger equation or compute the eigenvectors of the operator $n_x\sigma_x+n_y\sigma_y+n_z\sigma_z$ and dot it with the eigenvector of positive eigenvalue orthogonal to the other vector.
And yes there are easier ways to do this and more you can get out of it. But hopefully you see the other geometry.

Could you show how one obtains then the $cos \theta/2$ and $e^{i\phi}$ terms? 

I was using the Pauli spin operators, if you want to pick a basis you can write them as matrices (an operator is a function on a vector space, a matrix stands in for an operator after you select a basis; the operator exists and is the same regardless of what basis you may or may not select later).
$$n_x\sigma_x+n_y\sigma_y+n_z\sigma_z=\left(\begin{matrix} n_z & n_x-in_y \\ n_x+in_y&-n_z \end{matrix}\right).$$
And the eigenvector with positive eigenvalue is $\left(\begin{matrix} -n_x+in_y \\ n_z-1 \end{matrix}\right),$ unless $n_z=1$ then it is $\left(\begin{matrix} 1\\ 0 \end{matrix}\right).$ Let's deal with the case of $n_z=1$ first, in that case $a=1$ and $b=0$ and $\theta=0$ so $a=\cos(\theta/2)$, $b=e^{i\phi}\sin(\theta/2)$ all works out.
If you want to write the eigenvector as a unit vector you get $\frac{1}{\sqrt{2-2n_z}}\left(\begin{matrix} -n_x+in_y \\ n_z-1 \end{matrix}\right).$ If you want to adjust the phase so that the first coordinate is real and positive then you get $\frac{1}{\sqrt{2-2n_z}\sqrt{n_x^2+n_y^2}}\left(\begin{matrix}n_x^2+n_y^2\\ (n_x+in_y)(1-n_z) \end{matrix}\right).$
The rest is trigometry, e.g. $\frac{n_x+in_y}{\sqrt{n_x^2+n_y^2 }}=e^{i\phi}.$ So we just need to show that $\cos(\theta/2)=\sqrt{\frac{n_x^2+n_y^2}{2-2n_z}}$ and that $\sin(\theta/2)=\sqrt{\frac{1-n_z}{2}}.$ The latter is a trig identity $\sin(\theta/2)=\sqrt{\frac{1-\cos(\theta)}{2}}.$ 
The former is $$\sqrt{\frac{n_x^2+n_y^2}{2-2n_z}}=\sqrt{\frac{n_x^2+n_y^2+n_z^2-n_z^2}{2-2n_z}}$$ $$=\sqrt{\frac{1-n_z^2}{2-2n_z}}=\sqrt{\frac{(1-n_z)(1+n_z)}{2-2n_z}}$$
$$=\sqrt{\frac{1+n_z}{2}}=\sqrt{\frac{1+\cos(\theta)}{2}}=\cos(\theta/2).$$
