It is because the $\rm U(2)\times U(2)$ symmetry breaking is much smaller than the  $\rm U(3)\times U(3)$ symmetry breaking, not to speak about  $\rm U(6)\times U(6)$ symmetry which is badly violated by large masses of heavy quarks $(c,b,t)$. The characteristic scale of strong interactions is $1$ GeV (the proton mass), while the u,d quark masses are of order a few MeV. Thus the predictions of  $\rm U(2)\times U(2)$ symmetry for hadrons must be valid to an accuracy of about $1\%$. The s-quark mass is much larger (around 100 MeV), so  $\rm U(3)\times U(3)$ symmetry predictions are not nearly as accurate. Of course, chiral anomalies destroy the axial $\rm U(1)$ symmetry, so we should really be talking about  $\rm SU(2)\times SU(2)\times U(1)$ instead of  $\rm U(2)\times U(2)$ and $\rm SU(3)\times SU(3)\times U(1)$ instead of $\rm U(3)\times U(3)$.