Consider a molecule of oxygen in a balloon.
You know that at nonzero temperature all those molecules are bouncing around in all directions.
Of course, the mass of air doesn't have any net motion in any direction.
Indeed, the average velocity of each molecule is zero:
$$\langle v \rangle = 0 \, .$$
Of course, the average energy of any particular molecule is (from the equipartition theorem) $\langle E \rangle = (3/2) k_b T$.
We can rewrite this as
\begin{align}
\left\langle \frac{p^2}{2m} \right\rangle &= \frac{3}{2} k_b T \\
\left\langle v^2 \right\rangle &= \frac{3 k_b T}{m} \neq 0 \, .
\end{align}
The point here is that having zero average velocity definitely doesn't mean you have zero average squared velocity.
Zero average velocity just means you go left as often as you go right.
You can still be bouncing around all over the place.
You probably have an intuitive idea that there should be something with dimensions of velocity, not velocity squared, which represents the "typical" speed of a particle.
That's precisely what
$$\sqrt{\langle v^2 \rangle}$$
is.
Think of it like this: if half the particles are moving at velocity $v$ and the other half are moving at $-v$, then you get
$$\sqrt{\langle v^2 \rangle} = v \, .$$
So in this simple case the square root of the mean square velocity really is just the average speed.
You might think a simpler definition of "typical speed" is
$$\langle | v | \rangle$$
because this is literally the average speed.
In a sense this is simpler, but the root mean square is more useful because, as you can see from your own post, it's directly related to the energy.
Being related to the energy means that the root mean square velocity is also easily related to things like pressure, etc. which is why we often use it instead of the average absolute value of velocity (although both are useful).
Of course, this was all a classical picture, but in some ways the quantum case is similar.
For example, if you could cook up an experiment to directly measure the square momentum of the particle in the box, the result averaged over an ensemble of particles would be the result from the original post (I use velocity and momentum interchangeably as they differ only by a scale factor $m$).
In that sense, the expression calculated for $\langle p^2 \rangle$ is just exactly what it sounds like: it's the average you would find if you measured the squared momentum on an ensemble of particle in a box quantum systems.
Taking the square root to get $\sqrt{\langle p^2 \rangle}$ you just have, like the classical case, a measure of the typical momentum that happens to be really useful because it easily connects to energy, pressure, etc.
Of course, the average here is over an ensemble of quantum systems, so by "typical" we mean typical as averaged over the ensemble, not necessarily as averaged over time for a single particle.
There is at least one important difference though: a quantum system in the ground state has nonzero mean square momentum, but cannot give off any energy.
This is very different from the classical case where nonzero mean square momentum means there's internal energy which can be transferred to another system (i.e. measured).
This is related to the uncertainty principle.
Quantum systems have "fluctuations"$^{[a]}$ which are not thermal in nature.
$[a]$: Calling the fact that quantum systems have nonzero mean square momentum in the ground state "fluctuations" is a dangerous game because it can make the reader think those "fluctuations" are just like classical ones.
However, they are not.
For example, classical fluctuations have finite bandwidth because the underlying microscopic processes have internal time scales.
Quantum "fluctuations" do not.
What we typically call "quantum fluctuations" are really the manifestation of sampling a wave function as it shows up in the classical measurement apparatus.
