How is the virtual image reconstructed from a hologram? To make a hologram a film is exposed to an incident plane wave and wave from the object to record the interference pattern on the film. The principle is commonly explained in a way like that in p.1212 of "University Physics" ( http://books.google.com.hk/books?id=7S1yAgAAQBAJ&pg=PA1211&lpg=PA1211&dq )
What I don't understand is why a 3D image can be made by shining a plane wave through the film. The film is grating so at some points constructive interference can produce the point representing the object. But why the overall wave is diverged (show in 36.29b, p.1212) ?
 A: Because of diffraction. When the photographic plate is exposed, it blackens and change its refractive index in a spatially varying manner. When illuminated again by the reference beam it can be considered as an amplitude transmittance. In 2D you would define it as the complex function: 
$$t(x,y)= T(x,y)e^{i\theta(x,y)}$$ 
Let's say your reference beam can be approximated as a planar harmonic wave, it will have the form:
$$E(x,y)=E_0 e^{ik*r}$$
When you shine the reference beam through the hologram you modulate it with the transmittance. So, the field directly after the hologram is:
$$E(x,y)=T(x,y)E_i(x,y)$$
To get the wave field at any point farther than that formally, you need to solve the Fresenel-Kirchhoff integral. 
Diffraction results in a diverging and a converging wave fronts on both sides of the plate. The converging one creates a virtual image of the object that appears to stand where to original object was. The following illustration attempts to show this process:

A: I'd like to add another take on Cape Code's answer.
Holography works because, given reasonable physical assumptions, solutions to the Helmholtz wave equation are uniquely defined by the values of the solutions on one plane. So if we can light a phase / amplitude mask encoding a particular wave equation solution on a plane with a plane wave from a laser, the light passing through the mask will have the same phase and amplitude of the original solution to the wave equation. Therefore on propagating away from the mask, the light field will have the same behavior as the original solution.
To see this in action, let's assume that a light field is nearly monochromatic and nominally propagating in the $+z$ direction. Recall that plane wave solutions to the Helmholtz equation $(\nabla^2+k^2)\psi=0$ are of the form $\psi_{\vec{k}}(\vec{r}) = \exp(i\,\vec{k}\cdot\vec{r})$, where $\vec{k}=(k_x,\,k_y,\,k_z)$ is the wavevector fulfilling $c^2\,\vec{k}\cdot\vec{k}=\omega^2$. If the field comprises only plane waves in the positive $z$ direction (i.e. $k_z>0$) then we can represent the diffraction of any scalar field on any transverse (of the form $z=c$) plane by:
$$\begin{array}{lcl}\psi(x,y,z) &=& \frac{1}{2\pi}\int_{\mathbb{R}^2} \left[\exp\left(i \left(k_x x + k_y y\right)\right) \exp\left(i \left(k-\sqrt{k^2 - k_x^2-k_y^2}\right) z\right)\,\Psi(k_x,k_y)\right]{\rm d} k_x {\rm d} k_y\\
\Psi(k_x,k_y)&=&\frac{1}{2\pi}\int_{\mathbb{R}^2} \exp\left(-i \left(k_x u + k_y v\right)\right)\,\psi(x,y,0)\,{\rm d} u\, {\rm d} v\end{array}$$
To understand this, let's put carefully into words the algorithmic steps encoded in these two equations:


*

*Take the Fourier transform of the scalar field over the transverse plane $z=0$ to express it as a superposition of scalar plane waves $\psi_{k_x,k_y}(x,y,0) = \exp\left(i \left(k_x x + k_y y\right)\right)$ with superposition weights $\Psi(k_x,k_y)$;

*Note that plane waves propagating in the $+z$ direction fulfilling the Helmholtz equation vary as $\psi_{k_x,k_y}(x,y,z) = \exp\left(i \left(k_x x + k_y y\right)\right) \exp\left(i \left(k-\sqrt{k^2 - k_x^2-k_y^2}\right) z\right)$;

*Propagate each such plane wave from the $z=0$ plane to the general $z$ plane using the plane wave solution noted in step 2;

*Inverse Fourier transform the propagated waves to reassemble the field at the general $z$ plane.


If you can understand these steps you should be other see how the solution to Helmholtz's equation, i.e. the full three-dimensional scalar light field, is reconstructed from its values on the plane $z=0$. The latter of course is what a phase and intensity mask hologram encodes.
Sometimes an amplitude-only mask is used. So instead of a mask outputting a field of the form $A(x,\,y)\,\exp(i\,\Phi(x,\,y))$ it outputs a field of the form $A(x,\,y)\,\cos(\Phi(x,\,y))$. But this latter field can be written:
$$A(x,\,y)\,\cos(\Phi(x,\,y)) = \frac{1}{2}(A(x,\,y)\,\exp(i\,\Phi(x,\,y))+A(x,\,y)\,\exp(-i\,\Phi(x,\,y)))$$
which is the field $A(x,\,y)\,\exp(i\,\Phi(x,\,y))$ that we want superimposed on the phase conjugate field $A(x,\,y)\,\exp(-i\,\Phi(x,\,y))$. In this kind of holography, one arranges the lighting so that the wanted field and its phase conjugate are propagating at a highish angle relative to one another, so that they swiftly separate allowing one to view each field separately.
