If I have, for example a particle with $s = 3/2$ and $\ell = 2$, what are the allowed values of $j$? I'm slightly confused because I know that $j = \ell + s$, so surely there is only one allowed value?
The problem is that $\ell$ and $s$ are the eigenvalues of the angular momentum, but each value of them corresponds to multiple underlying states (unless the eigenvalue is zero). In effect the eigenvalue represents the magnitude of a vector angular momentum, but not it's direction, and the various different underlying states represent different possible values of direction.
If the spin and orbital angular momenta point in the same direction then you would expect them to add, but if they point in opposite direction you would expect them to subtract.