It is indeed possible for the E field lines to be non-orthogonal to the surface of a conductor with finite conductance.
Remember, if the E field is derived from a scalar potential, $\phi$, then the E field being perpendicular to the surface of the conductor implies that the surface is equipotential. So there are two ways to make the E field non-parallel to the conductor:
have $-\frac{\partial \mathbf A}{\partial t}\ne 0$ so that the E field is not derived only from a scalar potential
have $ \phi$ be spatially non-constant on the conductor surface so that even the static E field is non-perpendicular
Case 1) can be achieved using an external time-varying magnetic field, as you suggested. Case 2) can be achieved simply by having a current in a conductor with finite conductivity.
Here is a paper that describes a semi-quantitative approach for determining the surface charge on a conductor based exactly on how much the E-field “bends” at the surface.
https://www.tu-braunschweig.de/index.php?eID=dumpFile&t=f&f=138440&token=2cc8a71e4fdbf159121c6b8ef8348952a2e0c197A semiquantitative treatment of surface charges in DC circuits. Rainer Mueller. Am. J. Phys. 80 (9), September 2012. http://dx.doi.org/10.1119/1.4731722