What do we mean when we say that some Hamiltonian is invariant under rotations? 
Before going on to examine some examples of spin precession, it is worthwhile commenting on the time dependence of the expectation values $(4.23),(4.28)$, and (4.30). First, note from (4.16) that
$$
\frac{d}{d t}\left\langle S_{z}\right\rangle=\frac{i}{\hbar}\left\langle\psi\left|\left[\hat{H}, \hat{S}_{z}\right]\right| \psi\right\rangle
$$
We can see from the explicit form of the Hamiltonian (4.17), which is just a constant multiple of $\hat{S}_{z}$, that $\hat{H}$ commutes with $\hat{S}_{z}$ and therefore $\langle S_{z}\rangle$ is time independent [as (4.23) shows]. It is interesting to consider this result from the perspective of rotational invariance. In particular, with the external magnetic field in the $z$ direction, rotations about the $z$ axis leave the spin Hamiltonian unchanged. Thus the generator $\hat{S}_{z}$ of these rotations must commute with $\hat{H}$, and consequently from (4.31) $\langle S_{z}\rangle$ is a constant of the motion. The advantage of thinking in terms of symmetry (a symmetry operation is one that leaves the system invariant) is that we can use symmetry to determine the constants of the motion before we actually carry out the calculations. We can also know in advance that $\langle S_{x}\rangle$ and $\langle S_{y}\rangle$ should vary with time. After all, since $\hat{S}_{x}$ and $\hat{S}_{y}$ generate rotations about the $x$ and $y$ axes, respectively, and the Hamiltonian is not invariant under rotations about these axes, $\hat{H}$ does not commute with these generators.

How is one operator invariant (or not invariant) under another operation? I think the phrasing confuses me. What do we mean by "rotations about the z axis leave the spin Hamiltonian unchanged". Are we referring to the corresponding energy of the state and or are we just stating that the rotation operator and the Hamiltonian commute, i.e that the time evolved state of a rotated state is equal to the time evolved state rotated? Sorry for the unclarity.
 A: If you have a unitary $U^{-1} = U^{\dagger}$ (time independent) transformation such as $U(\varphi)=\exp\left[-\frac{i}{\hbar}\varphi\hat{S}_z\right]$ (the spin operator in the z-direction is the generator of rotations around the z-axis) then you transform the Hamiltonian via
$$H\to H' = UHU^{\dagger}$$
(This is analogous to a change of basis for a matrix). Now if the Hamiltonian commutes with the spin operator $[H, S_z] = 0$, then you can freely move the Hamiltonian around in the expression above since you can use the series definition of the exponential and then the Hamiltonian can move through every term. Then you'll see that
$$H' = H UU^{\dagger} = H 1 = H$$
so the Hamiltonian is invariant under rotations about the z-axis.
A: If $U$ is an unitary operator that acts on states in your Hilbert space and represents the operation (or operations) under consideration, then $H$ is invariant if
$$
H=U^\dagger H U.
$$
If you represent $U$ as a matrix of the same dimension as your Hilbert space, then $U$ transforms your basis vectors to a new basis, and thus $U^\dagger H U$ is the Hamiltonian expressed in this new basis.
What is confusing is that the title of your question refers to invariance under rotations, but the text refers to rotations about the $\hat z$ axis.  The rotation about $\hat z$ are in fact a subset of all possible rotations (there are rotations about $\hat x$ and $\hat y$, and in fact rotations about axes in arbitrary directions.)
In the example of your text, rotations about $\hat z$ are (probably) given abstractly as $U(\theta)=e^{-i\theta L_z}$.  Now, the exponential of an operator is given by its series
$$
e^{-i\theta L_z}=\hat 1-i\theta L_z +\frac{1}{2}\theta^2 L_z^2+\ldots
$$
so clearly if $[L_z,H]=0$ then then
$$
e^{i \theta L_z} H e^{-i\theta L_z}=H
$$
by expanding both exponential:
\begin{align}
e^{i\theta L_z}H e^{-i\theta L_z}= H+i\theta[L_z,H]-\frac{1}{2}\theta^2 [L_z,[L_z,H]]+\ldots 
\end{align}
(see  this part of wiki, although not the most digestible notation).
Conversely, if $e^{i\theta L_z}H e^{-i\theta L_z}=H$, then it must be (barring convoluted examples that rarely occur in physics) that $[L_z,H]=0$, which allows you to identify an operator that commutes with $H$.  Of course, if $A$ and $B$ commute with $H$, this doesn’t mean that $[A,B]=0$.
Finally, if you have a hermitian operator $A$ so that $[A,H]=0$, then you can construct the transformation $U(\alpha)=e^{-i \alpha A}$ and it will commute with $H$.  This is quite useful when one can easily identify such an operator.  Then $U(\theta)$ will generate a change of basis that will leave $H$ unchanged, i.e. it will generate transformations that will leave $H$ invariant.
Another example would be parity in a symmetric potential. Then $P^\dagger H P=H$ since $V(x)=V(-x)$ and the kinetic term likewise doesn’t change.  This allows you to divide your Hilbert space into even or odd solutions.
The difference between parity and rotations is that rotations depend continuously on an angle, so that one can write $U(\theta)=e^{-i \theta L_z}$ for some hermitian $L_z$, whereas there is no such exponential form for parity.  In other words, it’s not possible to recover something like $L_z$ that commutes with $H$ for parity, even if $H$ is invariant under parity and you can still divide the Hilbert space in even/odd sectors.
