The answer lies in the band structure of the two materials. The band structure describes how the electrons in a solid are bound, and what other energy states are available to them.
Very simply, the band gap for transparent diamonds is very wide (see [this link}(http://www.webexhibits.org/causesofcolor/11A0.html)
Normally, diamond is not a conductor: all the electrons live in the "valence band", and you need a photon with at least 5.4 eV of energy to push an electron into the conduction band. In the process, that photon would be absorbed. A photon with less energy cannot give its energy to an electron, because that electron "has nowhere to go". And since visible light has energies of between 1.65 and 3.1 eV, only UV photons have enough energy to be absorbed by pure diamond.
That same link also describes how impurities give rise to color in diamond: for example, nitrogen atoms produce an "intermediate" energy level, and this gives rise to more energetic electrons that could jump the gap to the conduction band and absorb light.
By contrast, graphite is a conductor. As a conductor, it has electrons in the conduction band already. You know this, because even a tiny voltage will give rise to a current - this tells us that the electrons didn't need to be "lifted" into the conduction band first. And since electrons will absorb any amount of energy easily, the material absorbs all wavelengths of light: which makes it black.