What are the reasons that $SU(3)$ is used for QCD?
Why wouldn't the simpler & smaller group $SO(4)$ make a better candidate?
What are the reasons that $SU(3)$ is used for QCD?
Why wouldn't the simpler & smaller group $SO(4)$ make a better candidate?
It is basically an experimental result that QCD is based on $SU(3)$. For example, we know that there are 8 gluons, no more no less. Otherwise several cross sections sensible to the color factors would be off (perhaps, the most recent example is provided by the Higgs boson production cross-section via gluon fusion measured by ATLAS and CMS at CERN). A theory based on $SO(4)$ has instead 6 gauge bosons. In fact, the theory would be based on $SU(2)\times SU(2)\sim SO(4)$ and could in principle admit two gauge couplings rather than one.
And notice that these are just two examples of several others reasons why $SU(3)$ is the correct group (at least at energies $E<1$~TeV).
I guess the simple answer is this: $SU(3)$ ist the SMALLEST group to allow for nonlinear effects (i.e. Confinement).
Side note: the fact that a model using $SU(3)$ is confirmed by experiment does not rule out other models based on other groups. But $SO(4)$ is already ruled out theoretically, because no confinement is possible.