I'm learning about the section of the Standard Model's lagrangian of Higgs-quarks interactions. This means writing a lagrangian made of a scalar field $\phi$ (the Higgs boson) and spinors $\psi$ (quarks) which is gauge invariant under
$$ SU(3)_C \times SU(2)_L \times U(1)_Y $$
I was told by my professor that the only possible terms with dimensions less or equal to 4 are $\overline{\psi}_L\psi_R\phi$ and $\overline{\psi}_R\psi_L\phi$.
I understand why $\overline{\psi}_L\psi_L\phi$ and $\overline{\psi}_R\psi_R\phi$ are not possible: they would violate the $SU(2)_L$ symmetry since $\phi$ is also a $SU(2)_L$ doublet; I also understand why $\overline{\psi}_L\psi_R$ and $\overline{\psi}_R\psi_L$ are not possible: their $U(1)_Y$ hypercharge is such that it doesn't equal zero after a $U(1)_Y$ transformation of these two terms.
What I don't understand is why $\overline{\psi}_L\psi_L$ and $\overline{\psi}_R\psi_R$ are not possible: both the $SU(2)_L$ and $U(1)_Y$ transforms should cancel out because both are unitary matrices. What am I missing?