Sorry for my poor english. My native language is french.
A general symmetry argument can be used:
*) If a system admits a symmetry plane (invariant by mirror symmetry), the electric field (true vector or polar vector) at a point on the plane is in the plane while the magnetic field (pseudo vector or axial vector) is orthogonal to the plane.
*) It is the opposite for an "antisymmetry" plane (system invariant by a mirror symmetry followed by a change of the sign of the charges).
In your example, the plane containing the axis of the solenoid and passing through a point M is an antisymmetry plane and therefore the electric field at point M is orthoradial.
These symmetry properties are contained in Maxwell's equations. They could be proved by using directly Jefimenko's equations as we do in statics with Coulomb's and Biot and Savart's laws.
But one can also justify them by invoking the invariance of electromagnetism under parity: One cannot distinguish right and left with an electromagnetic machine. And so, if we build a machine that is the image in a mirror of a first machine, it continues to function by remaining the image in a mirror of the first machine. The Lorentz force must therefore be transformed accordingly.
For a system which is its own image (symmetrical), this requires that the electric field at two symmetrical points is symmetrical and that the magnetic field is "antisymmetrical" (because of the vector product).
For an "antisymmetric" system: it only becomes identical to itself after a mirror symmetry and a change of sign of the charges which leads to a change of sign of the fields. Thus, at two symmetric points, the electric field is "antisymmetric" and in particular at a point of an antisymmetric plane of symmetry, the field is normal to the plane.