Voltage of a battery in a circuit with an infinite resistance I have a dilemma that I would like to share with you concerning batteries (without neglecting internal resistance), emf and resistance of a circuit.
To better visualize my question you may need to check these pictures:


-First, I'm trying to grasp the concept of why and how the voltage drop at the terminals of a battery depends on the resistance of the circuit.
-Second, knowing that $$\epsilon= V_b + V_c$$ (battery and circuit, respectively), we know that $$\epsilon=I(R+r)$$
Considering the extreme cases where $R=0$ and $R=∞$,
what I expected was that at $R = ∞$, the voltage would be 0, because there would be infinite resistance and zero current, and since $V=RI$, it would be $V= ∞ \times 0=0$ which makes absolutely no sense to me. Plus, an ideal voltmeter has an infinite resistance but it does not give a voltage reading of zero when its terminals are placed on the battery terminals. 
What am I doing wrong, and how do the charges on the poles of the battery act in each of these cases? 
I'm sorry I could not be clearer in how to propose the question but I hope that it is enough. I'm still learning so do not rely on any of my assumptions. 
 A: I think that you have faced a conceptual problem because you have used the relationship $V=RI$, rather you should have been using the definition of resistance $R = \frac V I$.
Then as $V$ goes up and $I$ goes down the value of $R$ increases.
Also you must be careful about $I=0$ (current equals exactly zero) and $I\rightarrow0$ (current gets closer and closer to zero).
When $R$ is large it might be better to redraw the circuit diagram with the resistor $R$ being replaced by a leaky capacitor.  With no measuring instruments the circuit diagram is as follows:

If you wait long enough the potential difference across the leaky capacitor will be $\frac{\mathcal{E}R}{(r+R)}$.  
As the resistance increases the potential difference across the leaky capacitor will get closer to $\mathcal{E}$.  
You can show that there is charge stored on the leaky capacitor and hence a potential difference across the leaky capacitor by disconnecting the leaky capacitor from the circuit and connected the ends of the leaky capacitor with a wire to find that a current flows in the wire.
Now measurement of the potential difference with a voltmeter could be tricky.

The reason being that the voltmeter has the potential of disturbing the circuit because it will have a resistance $R_V$ and so the current $i$ in the circuit will increase resulting in the potential difference across the leaky capacitor decreasing due to a greater potential difference across $r$.
The larger you make $R_V$ the closer voltmeter reading will be to $\frac{\mathcal{E}R}{(r+R)}$.
What I have tried to show is that what you can do using an abstraction is not necessarily realisable in practice.  So you can imagine $R$ getting larger and larger and using a voltmeter whose resistance is much, much larger that $R$ but it is another thing actually being able to do that in the real world.
A: 
-First, I'm trying to grasp the concept of why and how the voltage drop at the terminals of a battery depends on the resistance of the circuit.

The EMF, $\mathcal{E}$, exists chemically and is always present. If there is a path for current to flow, then there will be a voltage drop across the internal resistance, $r$, and the external resistance, $R$.  For short periods of use of the battery $r$ will remain constant. That means the current is regulated by $R$ via $$I=\frac{\mathcal{E}}{(r+R)}.$$ (Notice that $\mathcal{E}$ is not regulated by this equation.) The terminal voltage measured for the battery will be $\mathcal{E}$ dropped by the internal resistance voltage drop, $Ir$.  As $R$ changes, so does the current and the terminal voltage.
If the external resistance, $R$, becomes extremely large, $R\to\infty$, the current drops to zero and the internal voltage drop $Ir$ goes to zero, leaving $\mathcal{E}$ by itself as the terminal voltage.
$V=IR$ is very specific to individual resistors and really should be though of as $V_R=I_RR$, and doesn't (neglecting internal resistance effects) affect regulated or chemical voltage sources.
A: Bear in mind that infinite resistance is equal to an open circuit. In real life if you have a battery not connected to anything, there will always be a voltage across the terminals. Even a lead-acid car battery will produce sparks if you short the terminals!
A: At R→∞ voltage won't be 0 but little
less than EMF off cell, since a series
circuit with voltmeter and battery is
complete.
Yes, voltage across resistor reaches
0.Voltmeter reading is not EMF
because it draw some current which
pass through internal r and gets some
potential across it. 
An Ideal voltmeter cannot tell you
voltage across resistance in practical.
