Why Don't the Ladder Operators Commute? I have two problems with ladder operators.
The first is that I feel they should somehow result in measurable things. The asymmetry of applying the plus operator versus the minus operator is very strange to me. 
Second, I don't understand why applying the raising operator and then the lowering operator is different from applying the lowering operator and then the raising operator. The two are explained in terms of moving up and down energy levels, so shouldn't going up then down, or down then up, be the same operation? What is asymmetric about harmonic oscillators that causes this asymmetry? 
 A: Consider the simple harmonic oscillator Hamiltonian 
$$\tag{1} \hat{H}~:=~\frac{\hat{p}^2}{2m}+\frac{1}{2}m\omega^{2}\hat{x}^2
~=~\hbar\omega(\hat{n}+\frac{1}{2}),$$
where $\hat{n}:=\hat{a}^{\dagger}\hat{a}$ is the number operator.
Let us put the constants $m=\hbar=\omega=1$ to one for simplicity. Then the annihilation and creation operators are
$$\tag{2} \hat{a}~=~\frac{1}{\sqrt{2}}(\hat{x} + i \hat{p}), \qquad
\hat{a}^{\dagger}~=~\frac{1}{\sqrt{2}}(\hat{x} - i \hat{p}),$$
or conversely,
$$\tag{3} \hat{x}~=~\frac{1}{\sqrt{2}}(\hat{a}^{\dagger}+\hat{a}), \qquad
\hat{p}~=~\frac{i}{\sqrt{2}}(\hat{a}^{\dagger}-\hat{a}).$$
Eqs. (2) and (3) yield the identity 
$$\tag{4} [\hat{x},\hat{p}]~=~i[\hat{a},\hat{a}^{\dagger}]. $$
In other words, the non-commutativity of the ladder operators $\hat{a}$ and $\hat{a}^{\dagger}$ is directly related to the non-commutativity in the canonical commutation relation (CCR):
$$\tag{5} [\hat{x},\hat{p}]~=~i{\bf 1}\qquad \Longleftrightarrow \qquad [\hat{a},\hat{a}^{\dagger}]~=~{\bf 1}. $$
II) Thus if the ladder operators $\hat{a}$ and $\hat{a}^{\dagger}$ would commute, as OP ponders, then all operators of the theory would commute, all quantum mechanics would have been thrown out with the bath water, and we would be back doing classical mechanics.
Phrased equivalently, if the ladder operators $\hat{a}$ and $\hat{a}^{\dagger}$ would commute, then $\hat{a}$ (and $\hat{a}^{\dagger}$) would be a normal operator, and it would indeed be possible to associate $\hat{a}$ with a complex observables, cf. this Phys.SE answer. The corresponding two commuting self-adjoint operators would in this case be $\hat{x}$ and $\hat{p}$, which then could be measured simultaneously.
In conclusion: As we know that the ladder operators $\hat{a}$ and $\hat{a}^{\dagger}$ do not commute, then $\hat{a}$ is not a normal operator, and hence $\hat{a}$ is not a complex observable.
A: The ladder operators are not hermitian, so they are not measurable. Furthermore, by construction they do not commute, so there is no reason for them to commute... 
In addition, they are not just rising operators, as they multiply the Fock states by a number, on top of rising or lowering the number of bosons.
Try to apply them, expressed in function of $\hat x$ and $\hat p$, on any wave-function, and you will see what happens.
A: The reason that creation and destruction operators don't commute is that, on top of 'moving a state up and down energy levels', they multiply it by a number in the process, and this number depends on where you are in the ladder. More specifically, 
$$\begin{cases}
\hat{a}|n\rangle&=\sqrt{n}|n-1\rangle,\text{ while}\\
\hat{a}^\dagger|n\rangle&=\sqrt{n+1}|n+1\rangle.
\end{cases}$$
Thus, if you act on a number state $|m\rangle$ with a creation operator first, you will get the $\sqrt{n+1}$ factor with $m=n$, but the $\sqrt n$ factor with $n=m+1$, so you will get $\hat a \hat a^\dagger|m\rangle=(m+1)|m\rangle$. Conversely, if you act with the destruction operator first, you will get a factor with $\sqrt n$ at $n=m$, and then a factor of $\sqrt{n+1}$ at $n=m-1$, so the result will be $\hat a^\dagger \hat a|m\rangle=m|m\rangle$.
Ultimately, though, there is a more fundamental reason why creation and annihilation operators can't commute, and it is the fact that energy levels are bounded from below: that is, there are no $|n\rangle$ with $n<0$. To see why this is the case, consider the creation and annihilation operators without the prefactors I just talked about, to get
$$\begin{cases}
\hat{E}\,|n\rangle&=|n-1\rangle\text{ and}\\
\hat{E}^\dagger|n\rangle&=|n+1\rangle.
\end{cases}$$
The argument above no longer holds, and both routes will return the same coefficient on $|m\rangle$ after applying $\hat E$ and $\hat E^\dagger$ in both orders. The problem, however, is what happens to the ground state? The formulas above are mostly fine, but they do not specify what $\hat E|0\rangle$ should be, and there is no $|-1\rangle$ state we can put it into. However, if we want both operators to be hermitian conjugates, then we have really no room to play with, because the identity
$$
\langle n|\hat E|0\rangle^\ast=\langle 0|\hat E^\dagger|n\rangle=\langle 0|n+1\rangle=0
$$
implies that $\hat E|0\rangle$ has zero component along $|n\rangle$ for all $n\geq0$; since that is a complete set, it implies that $\hat E|0\rangle=0$. 
Finally, then, if you compute $\hat E\hat E^\dagger |0\rangle$ as above, you will get $|1\rangle$ as normal, but $\hat E^\dagger \hat E|0\rangle$ returns 0, which is different, and the operators don't commute:
$$\hat E\hat E^\dagger =1\text{ but }\hat E^\dagger \hat E=1-|0\rangle\langle 0|.$$
The only way around this is to allow for an infinite ladder of negative-energy eigenstates, which are completely unphysical, but do allow for commuting ladder operators. While these states have indeed appeared in the literature they are unwieldy to use as they defy all intuition and they have not caught on.
So, to answer your final question: the asymmetry in the harmonic oscillator that ultimately causes this behaviour is the fact that the ladder of energy eigenstates is bounded from below but not from above, which makes going up and down the ladder not completely equivalent.
