Difference between the expected value of the magnitude and magnitude of the expected value Suppose I want to make an experiment to measure the magnitude of the orbital angular momentum of an electron. When I take the average of my measurements, which out of the values below does it correspond to:
\begin{equation}\sqrt{\left\langle L^2 \right\rangle}~\text{vs.}~\left\langle \sqrt{L^2} \right\rangle~\text{vs.}~\sqrt{\left\langle \textbf{L} \right\rangle^2}?\end{equation}
To what measurements do these quantities correspond?
 A: I am going to start out by talking about linear momentum rather than angular momentum, and then explain why I have done this later. 
If I make a series of measurements $k_1,k_2,k_3...$ and want to take their average magnitude, then which of the above expressions I calculate correspond quite simply to the order in which I perform the operations the operations of averaging, squaring and square rooting. There is nothing magic here. I have a list of numbers and I am performing standard operations on them. Expectation values in QM are fundamentally exactly what you would expect from basic statistics.
\begin{align} \langle \hat{p} \rangle &= \langle\psi| \hat{p}|\psi\rangle\\
 &= \sum_k \langle\psi| k\rangle \langle k|\hat{p}|k\rangle\langle k|\psi\rangle\\ &= \sum_k k |\langle k|\psi\rangle|^2\\
&= \sum_k k \Pr(\mathrm{measured\;momentum = k}) \;.\end{align}
The one time when QM can introduce some subtly is when the observable you are measuring is composed of a number of other, non-commuting observable. For example the total (squared) orbital angular momentum $L^2 = L_x^2 + L_y^2 + L_z^2$ but since different components of angular momentum do not commute, they cannot be measured simultaneously. This means that we cannot measure $L^2$ by measuring its components separately and then combining them; we must measure it directly.
Since we cannot measure all components of $\mathbf{L}$ simultaneously, we cannot obtain a measured value for $\sqrt{\langle\mathbf{L}\rangle^2}$, but since $L^2$ is measurable we can experimentally obtain both $\langle\sqrt{L^2}\rangle$ and $\sqrt{\langle L^2\rangle}$, by repeatedly measuring $L^2$ and then square rooting and averaging or averaging and square rooting the results respectively. 
A: I will be working from your comment that states you are interested in a general operator. Thus, I will interpret that your question is that, if you have a vectorial operator, how to compute the magnitude of its angular momentum. For this, you actually thought in 3 possible operators: $L^2$, $\sqrt{L^2}$ and $\vec{L}$. Notice I ignored anything you put outside the $\langle \,. \rangle$. That is because these are not part of computing an expectation value, but rather operations you did with your results after measuring the operator. Looking in reverse order:


*

*In $\vec{L}$ you actually measured each component individually. The problem with this is that the expectation value of a given component might be 0, which does not imply it does not contribute with the expection value of the magnitude of the operator. Just imagine the angular momentum example, and a state which is an eigenstate of $L_z$ and $L^2$. $\langle \vec{L}\rangle = \langle L_z \rangle = m \hbar,\; -l < m < l$. However, the magnitude of the angular momentum will be $l(l+1)\hbar^2 > m^2 \hbar^2$. This is related to the fact that the components of the operator do not commute, as pointed out in the answer of "By Symmetry" above.

*When you do $\sqrt{L^2}$ you are actually saying that your operator is the series $$ 1 + \frac{L^2 - 1}{2} - \frac{ (L^2 - 1)^2}{8} + \cdots$$
Now, you want to apply this series on your state and cross fingers for it to converge (It may not if the eigenstate of your operator falls outside the radius of convergence of the series). However, when it converges, it will give you the right answer.

*The correct answer is then choosing $L^2$. And it is easy to see why: You actually computed the magnitude operator and are them asking what is its expected value. As an example, if your system is in an eigenstate of the angular operator, it will give you the expected answer $l(l+1)\hbar^2$.
So, basically, one could have special cases where $\sqrt{\langle L^2 \rangle} $ and $\langle \sqrt{L^2} \rangle$ gives you the same result. But you will do extra math for nothing, and the $\sqrt{\langle L^2 \rangle} $ will always work.
