Orientation of the $2p$ states in a Neon atom Consider a Neon atom. It has 10 electrons: two in the $1s$ state, two in the $2s$ state and six in the $2p$ state. The six electrons in the $2p$ state have a wave function which is not spherically symmetric, but possesses an orientation in space. Two point in an $x$-direction, two in some $y$-direction and two in some $z$-direction. This is explained in textbooks on QM as a consequence of determining the set of orthogonal eigen-states of the atom. That is an explanation which I fully accept as the correct one. 
However, it occurs to me as somewhat peculiar that an atom like Neon --which is clearly spherically symmetric-- is somehow able to settle into a state with specific x,y,z orientations for its 2p electrons. Is this a random choice that is made when the Neon atom is created? Is the choice permanent, i.e. if one observes the atom now (t=0) and then much later at t=T, are the three orientations still pointing in the original direction? Or is it that the three orientations are not observable?
 A: The trick is that you can't actually tell one electron apart from another - they're identical particles, so it doesn't really make sense to say "this one occupies this particular orbital while this other one occupies this other particular orbital". There's no guarantee you're not going to mix them up between measurements.
Also, since all three $p$ orbitals are degenerate for an isolated neon atom, you can have arbitrary superpositions of the three $p$ orbitals without raising the energy from the ground state. So there's really no reason to believe that one particular electron occupies one particular orbital.
A: 
Two point in an x-direction, two in some y-direction and two in some z-direction.

This is not a physically useful choice of the basis. It does work for the $p$ shell, because there's only three possible values for $m$—same as number of directions in space—but already doesn't for the $d$ shell, where there are $2l+1=5$ possible values for $m$—not even a multiple of $3$.
In physics a more general approach is usually taken, where there's only one special axis, $z$, along which the spherical harmonics are aligned. Then, instead of $|p_x\rangle$, $|p_y\rangle$ and $|p_z\rangle$ orbitals of $p$ shell we get $|m=-1\rangle$, $|m=0\rangle$ and $|m=1\rangle$ orbitals.

However, it occurs to me as somewhat peculiar that an atom like Neon --which is clearly spherically symmetric-- is somehow able to settle into a state with specific x,y,z orientations for its 2p electrons

An atom settles in a particular state when it's prepared in such a state. For example, you can put your neon atom in a magnetic field (which is the very thing that makes $z$ axis special here), which would split energies of the different-$m$ states (Zeeman effect). After this, if you wait long enough (or somehow help the atom get into its ground state), the atom will end up in its lowest energy state, which is no longer degenerate in $m$. After this you can slowly remove the magnetic field, and you'll get your "$z$-oriented" atom.

Is the choice permanent, i.e. if one observes the atom now (t=0) and then much later at t=T, are the three orientations still pointing in the original direction?

If we are talking about an atom in free space, without any external influence, then, due to being an eigenstate, this initial state will remain unchanged for all $t>0$.
