You are not missing anything: no nature law enforcement agency would stop you from writing terms such as this. It would be Lorentz and made gauge invariant. You are presumably asking what observable consequences such a mixing term would entail. In the absence of other recondite couplings, none.
One assumes you also have the conventional diagonal kinetic and unequal-mass terms: the definition of $\psi_e$ and $\psi_\mu$ follows from the propagating mass eigenstates. If the masses were the same, you'd diagonalize the kinetic term which includes your off-diagonal proposal and its h.c., consistently with the (identity) mass matrix in the space of these two states.
If the masses are not the same, as in our world, diagonalizing the kinetic term would then throw the mass matrix off-kilter, and would introduce mixing terms at the mass level, $\epsilon ~\overline \psi_e \psi_\mu$ + h.c. Moreover, you'd then have to adjust your normalizations of the canonical fields in the diagonal kinetic term, to ensure the term is the identity matrix! This adjustment would further alter the newly off-diagonal, but still symmetric, mass term.
This mass term may now be diagonalized by a different orthogonal transformation than the previous one, by dint of the normalization adjustment indicated, and the e and μ masses will be shifted as a result of this adjustment. But the effect of this second rotation on the identity kinetic term will not be visible, of course. You will then have a Lagrangian with diagonal kinetic and mass terms, pretty much as in the non-mixed case baseline you tweaked, except with modified lepton masses. Can you estimate the ε-order of their tweak?
I gather you don't need explicit formulas implementing the above description, a useful exercise if you have never done such, and you pursued particle physics, where you wake up and go to sleep diagonalizing mass matrices.