Why isn't there a potential difference across a disconnected diode? I know this question sounds silly, as if there was a potential difference a current would be created when the terminals are connected together and this would mean energy has come from somewhere.
The reason I ask this though is that from my understanding of the depletion region and built in potential of a diode it seems like if you connected a voltmeter across the whole diode it would show the value of the built in potential.
This is explained in the image below:

At first, electrons flow from the n type to the p type because there are a higher concentration in the n type, and holes do vise versa. This is called the diffusion current. The first electrons and holes to cross the pn boundary are the ones which are closest to it; these carriers recombine when they meet each other and are then no longer a carrier. This means there is a depletion region of no carriers near the pn boundary. because electrons have left the n type material, and holes have left the p type material, there is a surplus of positive and negative charge on the n and p side of the pn boundary respectively. This causes an electric field that opposes the diffusion current, and so no more electrons or holes cross the boundary and combine. In short, only the electrons and holes near the boundary combine, because after they have done that an electric field is formed that prevents any more carriers from crossing. The current due to this electric field is called drift current, and when in equilibrium this will equal the diffusion current. Because there is an electric field at the boundary (pointing from the positive charge to the negative charge) there is an associated voltage. This is called the built in potential.
If you sample the electric field at each point along the diode from left to right, you would start with 0 in the p region because there are an equal number of protons and electrons. As you approach the depletion region you would see a small electric field pointing back towards the p region, caused by acceptor impurities which now have an extra electron (due to recombination) and therefore now have a net negative charge. This electric field would increase in strength as you get closer to the boundary, and then die away as you get further away.
This electric field means there is a voltage, as shown in graph (d). The p side is at an arbitrary potential, and the n side is at a potential higher than this because there is an electric field between them. This means there is a potential difference across the depletion region; this is known as the built-in potential.
But why when I connect a volt-meter across the whole diode will I not see this built in potential?
Edit:
I've also asked this question on another SE site And they say that the reason is because the net potential difference across the diode once it has been connected to a wire or volt meter is zero, because a metal-semiconductor junction has an electric field between it, and the overall effect of the electric fields at the junctions of the m-p-n-m materials results in zero potential difference.
Regardless of the fact that voltage does not equal electrostatic potential difference, can you please confirm that there is also no net electrostatic potential difference across a diode connected to a wire or volt meter because of the m-s junctions? I'd like to get a physicists answer on this, to see if this is the truth or just a simplified explanation to make it easier to accept.
 A: 
But why when I connect a volt-meter across the whole diode will I not see this built in potential?

Because at equilibrium (V=0) the Fermi-level throughout the whole device is flat. A voltage will only exist when there is a gradient in Fermi-level over the component (i.e. a voltage drop).
Edit
Maybe a more visual explanation would help? Think of the Fermi-level as the height of water in two buckets. The first bucket on the left has a very low water level, the second bucket on the right has a very high water level. P-doped semiconductor have a Fermi-level close to the valence band so this is represented by the left bucket. N-dopded semiconductors have a Fermi-level very close to the conduction band, this is represented by the very high water level in the right bucket. We now connect the buckets together with a pipe and the water level equilibrates.
|        |         |~~~~~~~~|           |        |         |        |
|        |         |        |           |        |         |        |
|        |         |        |   --->    |~~~~~~~~ ========= ~~~~~~~~|
|        |         |        |           |        |         |        |
|~~~~~~~~|         |        |           |        |         |        |
|________|         |________|           |________|         |________|

Now obviously semiconductor band structure is not a bucket and water is not an electron gas but this illustrates the basic process. Let's try and stretch the metaphor a little... Before we connected the buckets they were both electrically neutral, this implies that by allowing the exchange of water we should expect an internal build-in electric field and associate potential.
But notice, that the "voltage drop" across the buckets is just the potential energy difference between the left and the right sides, which is zero because both waters levels have the same height.
So back to your question; we know the pn-junction has a built-in potential, so why can't we measure that? Well this is the potential required to lower the n-side band such that the Fermi levels of both sides can align. The internal potential is equal to the difference of Fermi-levels (water heights) before alignment.
A: To address the follow up question, which requires more characters than a comment allows:
In a simple idealized view, the Fermi level is the top energy level in the solid occupied by electrons. In silicon with no doping it sits at mid-gap: the valance band is full, the conduction band empty. In a thought experiment, if you had two separate chunks of intrinsic silicon each would be perfectly happy in isolation. If you could mash them together to make a "junction", everything would still be perfectly happy - the Fermi levels line up, and no electron has any real desire to do something else.
Adding dopants shifts the Fermi level. In n-doped material there are newly available occupied levels near the conduction band edge, and in p-doped materials there are newly available un-occupied levels near the valance band. By doping the material you have fixed the Fermi level at a new point. In isolation, the n-doped chunk is happy, and the p-doped chunk is happy.  Here, though, when you mash them together to make the junction, they realize that together they aren't happy. The occupied levels on the n-doped side are above empty levels in the p-doped side: this is a non-equilibrium condition since electron (hole) flow will reduce the energy of the system. But wait!  Indeed, the electrons (holes) start sloshing around, but in doing so they leave behind the dopants.  These now-abandoned dopants have a net charge, and an internal field begins to build up.  It will build up until it is large enough to prevent further electron (hole) flow across the junction.  However, an equivalent view is that this built in voltage is just what is needed to bring the different Fermi levels into alignment (as in @boyfarrell's picture above). (I find the water analogy a bit misleading, since it is the separation of fixed dopants and moving charges that leads to the build-in voltage.)   
So, ultimately, yes: the built-in voltage is precisely what makes the p-n junction in equilibrium with no net potential across the terminals.
Metal-semiconductor (as well as hetero-semiconductor) junctions can be trickier.  In the simple picture, jamming silicon to silicon is easy - the valance band and conduction band will line up with each other seamlessly, the only question is where the relative Fermi levels are.  In a hetero structure (metal-semiconductor, GaAs-InGaAs, ...) it is possible that there are band offsets (valance bands don't line up), Fermi level pinning (defect states at the interface determine the relative offsets) and other effects that come in to play.  But, if the stars align properly, a metal-semiconductor junction can also have a built-in voltage produced, again sufficient (and necessary) to stop the flow of carriers between the two sides.
