The main reason is that boiling requires rather enormous amounts of energy.1,2 How many minutes does it take to bring, say, a quart or a liter of water to the boil (5 minutes?), compared to actually evaporating it by boiling (30 minutes?). That ratio is a good estimate for the energy it takes for water to evaporate. The energy for evaporation is drawn from the heat in the environment, which is frequently used when cooling is needed in technical and everyday applications.
So the order of events is:
The constant influx of heat from the flame keeps the water boiling.
The heat influx stops because you turn off the stove.
For a small moment there is enough heat in the vessel to boil some more water, which cools down the water proper and the vessel.
Because of the energy used for evaporation the water temperature drops below the boiling point, and the water stops boiling. This happens fast because the heat energy stored in the water and the vessel is small compared to the energy needed to evaporate water.
One could perhaps add that water can evaporate very quickly and basically uses the available energy "immediately". The physics are quite interesting because the limiting factor is how fast you can transport the energy into the water. The heat transport from a solid surface (read: metal pot) to liquid water is very good, partly because of the water convection. But when it begins to boil the vapor gets in the way which conducts heat comparatively badly, which leads to the funny effect that there is an optimum temperature for a heating surface above which the heating actually slows down until energy transfer by radiation takes over.
I suspect that one could evaporate a smallish amount of water "instantly" by radiating it with intense microwaves which transport the energy right where it's needed, circumventing, so to speak, all the heat transport issues.
1 Evaporation is a phase transition from the fluid phase ("water") to the gaseous phase ("vapor" or "steam", but be careful with everyday words — water vapor is invisible!). Most phase transitions involve large energy flows. That's the reason ice is an excellent coolant at 0 C (e.g. in your cooler). It's not only the temperature of the coolant, it is the energy needed for the imminent phase transition.
2 This energy is called the "(latent) evaporation heat" or enthalpy. For water it is about 2.2 Kilojoule per gram, if we can trust wikipedia, which would make it 2.2 Megajoule per liter, a.k.a 2.2 Megawattseconds (don't you love metric units?). For Watthours instead of -seconds we divide by the 3600 seconds in an hour3 to get 611 Watthours, which meets my guess of 30 minutes with a 1 kW cooker pretty nicely :-).
3 Too bad that hours don't have 100 minutes and a minute does not have 100 seconds!