Can a metallic rectangular waveguide support hybrid modes?

Studying the basics of guided waves in metallic rectangular waveguides, filled with homogenous dielectric material, I see that there is always a distinction between TE and TM modes. I understand that a TEM mode is impossible in such a configuration, but I'm wondering if hybrid modes (i.e. modes with both longitudinal components of the electric and the magnetic field nonzero) are possible and allowed by Maxwell's equation.

What really determines the mode structure is the boundary conditions, which in a metal wave guide are particularly simple: the electric field should be zero in metal, that is the component of the electric field parallel to the wall of the wave guide should be zero on the wall. Apart from that the waves are like the electromagnetic wave in vacuum - strictly transversal.

• Well, it is not possible that both the electric and the magnetic field are transversal in metallic rectangular waveguides. You can either have a TE or a TM mode, as far as I'm concerned. The question is if hybrid modes are also possible. Commented Apr 30, 2020 at 16:54
• What kind of hybridization do you mean? Commented Apr 30, 2020 at 17:25
• I mean modes in which both longitudinal components of the electric and the magnetic field are nonzero Commented Apr 30, 2020 at 17:35
• I understand, but how do you get modes like this? In optical wave guides it happens primarily due to the way the modes are reflected at the boundaries - there is an evanescent wave penetrating outside of the guide, and also due to the non-linearities of the material. None of these are present in metallic wave guides. It is also worth noting, that metallic wave guides historically appeared much before the optical fiber, which is where TM and TE terminology comes from - because it was a convenient and complete way to classify the modes. Commented Apr 30, 2020 at 17:40
• I don't know if it is possible to get modes like this in metallic waveguides, that's the point of the question. Whether it is possible or not, I'd like to see the mathematical proof Commented Apr 30, 2020 at 18:56

In a homogeneous guide for a hybrid TE&TM mode to exist you must have the propagation factors be equal. As an example take a rectangular guide of dimensions $$a\times b$$ then the $$TE_{mn}$$ and $$TM_{mn}$$ modes cutoff wavenumber is $$\kappa_{m,n} = \sqrt{ \left( \frac{m\pi}{a}\right)^2 + \left( \frac{n\pi}{b}\right)^2}$$, and their respective wavenumbers $$k_{mn}(\omega)$$ are equal to $$k_{mn}=\sqrt{\left( \frac{\omega}{c}\right)^2-\kappa_{mn}^2}$$. Still one could not call this a genuine hybrid mode because the amplitude of each mode can be independently chosen from that of the other.

For a circular guide of radius $$a$$ the $$TE$$ and $$TM$$ modes do not even have the same wavenumber. In fact, the cutoff modes have $$\kappa_{mn}(TE)=\frac {p_{mn}'}{r} \\{\kappa_{mn}(TM)=\frac {p_{mn}}{r}}$$ where $$J_m'(p_{mn}'a)=0$$ and $$J_m(p_{mn}a)=0$$. For certain combination of indices these may equal, for example $$p_{04}'=p_{14}=13.324$$, but again you have the same independence of the their amplitudes. So this is not a true hybrid mode.

In general, in a homogeneously filled waveguide the TE and TM modes have independent amplitudes and even if one can select a certain a pair of equal wavenumbers their combination is not a true mode. This is completely analogous to the distinction one can make between a pure state that is the energy eigenfunction of a quantum mechanical system and a linear combination of such states even in the case of a degenerate eigenvalue.