This is an example of 2+2=5.
First of all, not all reversible processes are isothermal. You have reversible isobaric, isochoric and adiabatic processes.
For a reversible adiabatic process, the temperature of the surroundings is irrelevant.
For a reversible isobaric and isochoric process, the temperature of the system and surroundings are always in equilibrium (actually, differ infinitesimally) but they are not constant.
As far as irreversible heat transfer processes are concerned, the temperature of the system and surroundings is not the same, except at the boundary. Within the system temperature gradients exist.
Finally, with regard to the (essentially) constant temperature surroundings, the mass or heat capacity need not be "infinite". It need only be large enough so that, given the amount of heat transfer involved, its change in temperature would be infinitesimal.
You say "For a reversible isobaric and isochoric process, the
temperature of the system and surroundings are always in equilibrium
(actually, differ infinitesimally) but they are not constant." But
isn't that contradicting your last statement that the temperature
change of the surroundings is infinitesimal i.e the temperature change
of the surroundings is not constant?
No. Because for the reversible isobaric and isochoric processes the surroundings is an infinite series of reservoirs ranging in temperature between the initial and final temperature, not a single reservoir as in an isothermal process.
Consider the reversible isochoric heat addition. The temperature of each reservoir is infinitesimally greater than the previous reservoir and infinitesimally greater than the gas temperature. So essentially we have $T_{sys} = T_{sur}$ = constant for each reservoir, meeting your criteria 2, but the system and surroundings temperature is not constant over the entire process because there are multiple reservoirs. So it is not an isothermal process.
Hope this helps.