If you drop a really heavy ball the ball's gravitational potential energy will turn into kinetic energy.
If you place the same ball in the pool, the ball will still fall. A lot of kinetic energy will turn into thermal energy because of friction, but the gravitational potential energy will still be converted.
Similarly, why doesn't electricity flow without a good conductor? Why won't Electrons flow from the negative terminal to the positive terminal without a wire attaching them?
Electricity flows like a wave and metals have free electrons in the electron cloud that allows the wave to propagate, or spread. But when these free electrons aren't available to propagate the wave, why don't the electrons just "move" like the ball? Why don't the electrons just "move" through the air to the positive terminal?
A slow drift speed means that the electrons most likely will take a long time to propagate the wave of electricity, but they should still get there.