For gauge groups like $SU(2)$ and $SU(3)$ etc. we know that observable states such as mesons or baryons must be charge neutral. However, for a $U(1)$ gauge group we can have charged initial states in our scattering experiments. Why can we have states with an observable non-zero charge for the $U(1)$ but not for $SU(n)$?
Another way to state this question, to clarify, is the following: the global part of the $U(1)$ gauge group defines a good quantum number - the total charge - which allows us to split up our Hilbert space. Why does the global part of bigger gauge groups not supply a good quantum number? If it does supply such a number why must all states be charge neutral under it?
Having had a discussion with @user1504 and taking into account @FredericBrunner 's points I think I can answer this question to my satisfaction.
The summary is thus - it is possible in principle, regardless of the gauge group, to form localised lumps of the respective charges under the global part of the group. To me, this gives the global part of the group a meaning separate from the gauge part. For $U(1)$ the lumps can have positive and negative charge and for $SU(3)$ they can have each of the three colours. This means that in respect of the question, $U(1)$ and $SU(N)$ are not different. In a practical sense however, the $SU(N)$ groups are confining so one can only do this in principle at a temperature sufficiently high that the theory is deconfined and the notion of individual quarks makes sense. Similarly, $U(1)$ in 2+1 dimensions is confining and there the notion of lumps of positive and negative charge only make sense above the deconfinement transition.