How can there be a voltage when there is no current? I'm told at school that the Electromotive Force (e.m.f) of a battery equals the potential difference between the terminals of the battery when there is no current.
How is that possible? How can there be a potential difference with no charge flowing?
 A: The analogy of electricity to flowing water may come in handy here. In this analogy, a potential difference is like a difference in height. One lake on top of a mountain and another  in a valley, for example, might represent the two terminals of the battery, which are at different potentials. If you think about that situation, it's clear that no water flows from the upper lake to the lower one because there's no path for it to get there. The same goes for current: when there's no path from the negative terminal of the battery to the positive terminal, current won't flow.
A: Another useful analogy, apart from the gravity one described by David Z, is temperature. You can think temperature as your potential, and the heat flow as your current. Two points of space may be at different temperature, but if they are correctly insulated, they won't exchange heat. The heat will flow only if they are connected somehow. For the current is the same: negative charges go from low to high potentials, if there is a suitable way to go through!
A: If you insert a dielectric in a circuit, you will not see any current but obviously there is a potential difference across the dielectric. To have a potential difference, you just need an electric field inside the material. This electric field might drive a current if the charges are mobile. 
A: Common misconception:  "Voltage is caused by current."   Nope, wrong.
In fact, currents are caused by voltage, since at the micro scale, the acceleration of charge carriers is caused by e-fields.  Voltage (or equipotential surfaces) is one way of describing e-fields.
The above misconception is extremely widespread, and deeply embedded in the tech culture.  I've met engineering grad students who firmly believe that current causes voltage.  They go partially (and humorously) insane when asked to explain the fact a capacitor sitting on a shelf can have a large voltage across its terminals.  Large voltage at zero current!  Blasphemy!!!
So, if voltage isn't caused by current, what then causes voltage?  Simple: electric charge causes voltage, since electric charge is permanently associated with e-fields, and voltage is simply a description of e-fields.
The misconception about current causing voltage seems to have a specific origin.  In the time before Faraday, all physicists believed in Instantaneous Action at a Distance.  When a rock fell downwards, it wasn't because of a gravity field.  There were no "fields.  Instead, rocks fell because of "Distant Action," or direct unexplained force.  And, if oppositely-charged objects attract each other, physicists sneered at the idea of "fields" in the space between objects.  Attraction was caused by Action at a Distance.  Michael Faraday was ridiculed for his proposed "EM fields," and in fact died without ever seeing his fields-concept adopted.  It took JC Maxwell to force the issue, and a few decades later the EM fields took over all of Classic physics.
And yet the "Action at a Distance" concepts never quite died.  As a result, most students are never taught that Voltage spews out of wires and hangs unsupported in empty space.  This fact is Faraday's great contribution to physics, but some textbook authors never learned it.  Instead they reject Faraday's disgusting "fields" concept, and they focus only on the potential energy of charges.  The PE description comes from the age of Action at a Distance.  Authors trapped in pre-Faraday thinking are weird.  They'll tell you that voltage is really just potential energy.  Well, "potential energy per unit charge," and they'll fiercely defend this misconception against any invasion of Faraday's "fields" concepts.
No, voltage is not potential energy.  The trouble is with that "unit charge" bit at the end.  Voltage can exist without any "unit charge" being present, and without any P.E. being stored as the unit charge moves across a certain voltage.  Therefore, voltage cannot be "energy per unit charge."  (This really isn't a complicated concept, for example Gravity is still above the ground, even when no small rock is being lifted in order to store some Potential Energy.)
So, to define Voltage we have no need of potential energy or unit charges.  Actually it's all backwards, and if we have voltage and unit charges, we can use this to define Potential Energy!  But knowing all about PE does not help us understand the nature of "Electric Potential."

Perhaps the following simplification might help:  e-fields or "electrostatic" fields are entirely made out of voltage!
This is similar to saying that e-fields are made out of flux lines.
When an e-field exists, the voltage is found in the space around the charged surfaces, and is everywhere perpendicular to the flux lines.  A charged object is surrounded by invisible lines of radial flux, but a charged object is also surrounded by invisible onion-layers made entirely of Electric Potential or voltage.  E-fields are really made of lines of force, but also they are really made out of pure voltage.
