Set aside the battery part, and think about what you get when you e.g. inject electrons into a block of insulating plastic. It looks like this:
That's a "beam tree", a.k.a. a "Lichtenberg figure". It's made by using an accelerator to deposit electrons in a block of plastic. A lot of electrons. They stay where they stopped, building up charge in the plastic that can't move, until a mechanical force (often a grounded nail hit by a hammer) at the 'trunk' end causes a bit of damage that can allow the charge to cascade out. Before that, there's a lot of electrostatic repulsion (it's a lot of charge), but no charge motion.
At the micro level, in the the pure plastic the electrons find themselves in a place where there are no (energy-accessible) states that correspond to being able to move. They're a perturbation, appear as a local defect, but there are no moving states available to them, so they can't move. They're stuck in a landscape of little pockets, without the energy to move out of their local pocket.
When the impact creates lots of additional defects, however, the game changes. Mechanical stress from the impact, localized heating from the bulk motion of changes, etc cause other possible states to appear. Some of those are energetically favorable ("I can finally get out from between these atoms through that gap!") and the electrons move under their own large potential.
This is somewhat different from the battery and circuit case, where you're trying to promote existing electrons via an electric field. But it shows quite clearly that the insulation effect is not due to a lack of movable electrons: Even extra "free" electrons can't move.
More formally, this is called the "localization" of the electrons: They've caught. In pure (i.e. geometrically perfect crystals materials) the general theory dates back to the 1964 "Theory of the Insulating State" paper by W. Kohn.
In this paper a new and more comprehensive characterization of the insulating state of matter is developed. This characterization includes the conventional insulators with energy gap as well as systems discussed by Mott which, in band theory, would be metals. The essential property is this: Every low-lying wave function 4 of an insulating ring breaks up into a sum of functions, $ \Phi = \Sigma \Phi_M$,which are localized in disconnected regions of the many-particle configuration space and have essentially vanishing overlap. This property is the analog of localization for a single particle and leads directly to the electrical properties characteristic of insulators. An Appendix deals with a soluble model exhibiting a transition between an insulating and a con- ducting state.
The localization process is a spatial one, not an energy one. It's possible to see it via the energy and momentum structure, but that's not the clearest way.
"Localization" as a cause of insulating behavior appears across insulating materials. (There are a few specialized insulators which have delocalized, i.e. broadly spread and/or overlapping wave functions, but they're really exotic cases) Since the 1964 paper there's been a lot of work on different forms of insulators, localization due to defects, impurities, etc, but the idea of "localization, i.e. locally-contained wave functions, preventing charge motion" remains a key paradigm.
