I've been thinking about this, and my answer is leaning towards yes, but I couldn't find any definitive answer by Googling.
If the higher potential side of a typical battery is connected by wire to a spike driven into the ground, will it eventually deplete?
My thought process is that the voltage difference will cause current to flow between the electrode and the earth, causing the battery to eventually "die," just as if it was connected in a circuit.
However, I also wonder what the difference is between this situation and shorting the battery. Connecting the anode and cathod of a battery together will cause it to heat up and damage itself, but will the same happen if its connected to ground? Isn't it (physically) the same situation?