I think I can answer in the case of the circular and square ring.
Firstly, by cylindrical symmetry, we know that the E field is the same all the way around the circle.
Why there is no radial component: Imagine rotating the system by 180 degrees about any diameter. At any location, the direction of the magnetic field would reverse, the direction of the current would reverse, all components of the E field would reverse, EXCEPT for the radial one. Now let's superimpose the two systems on each other. Without a magnetic field, we nevertheless have an induced radial E field! As this is clearly impossible, any radial component must be ruled out.
Why there is no component in or out of the page: Such a component would pull the ring into or out of the page. All the work put into changing the magnetic field goes into the work the electric field does in driving charges around the circuit. So it seems the ring's kinetic energy would increase without any source.
Similar arguments can be used to shown that in the case of the square ring, the induced electric field must once again be such that it is parallel to the ring at each point and drives around a current in it (direction determined by Lenz's law).
In general my intuition is that the induced electric field should be parallel to the wire at each point... but without these symmetry arguments in a general case I don't know how to go further.