I asked Mark Srednicki about this, and he told me that it's not really correct to say that negative-norm states break unitarity, because negative-norm states don't exist by the definition of the inner product. It's often a convenient calculational trick to formally expand your state space so that it's no longer a Hilbert space by adding in negative-norm ghosts, and the presence of physical states formally appearing to couple to ghosts indicates the presence of a quantum anomaly that prevents you from consistently quantizing your theory. But this is just a calculational trick to see the anomaly - the anomaly is real, the ghosts aren't.
In particular, you can always in principle see the existence of the anomaly without introducing ghosts. For example, the usual explanation for the fact that bosonic string theory can only be formulated in 26 dimensions is that that's the only number of dimensions in which the ghosts decouple. But we can alternatively work in light-cone gauge with only positive-norm states, and we find that only in 26 dimensions do the Lorentz generators close. This is another way to see the anomaly that doesn't require any mention of ghosts.
Mark also said that another reason it's incorrect to say that ghosts "break unitarity" is that they really just prevent you from consistently quantizing your theory at all - there's no reason to specifically single out unitarity as being broken.