Density Functional Theory (DFT) is formulated to obtain ground state properties of atoms, molecules and condensed matter. However, why is DFT not able to predict the exact band gaps of semiconductors and insulators?
Does it mean that the band gaps of semiconductors and insulators are not the ground states?
This is an important question that is asked by many people entering the field of density functional theory. I think that it should be answered with a high degree of detail and thus I would like to add a few aspects to the answer of supermarche.
It is well known (see, e.g., L. J. Sham, M. Schlüter: Density-Functional Theory of the Energy Gap, Phys. Rev. Lett. 51, 1888 (1983)) that the fundamental band gap for a system with $N$ electrons is given by the differences of ground-state total energies of systems with deviating numbers of electrons as $$E_g = (E_{N+1} - E_N) - (E_N - E_{N-1})$$ Thus being able to calculate the ground-state total energy for these different systems should be enough to calculate the band gap. Leaving aside the issue of the approximation to the exchange-correlation (xc) functional the ground-state total energy is accessible by density functional theory but this does not imply that the band gap of the Kohn-Sham system is the fundamental gap of the interacting-electron system.
Let us assume fractional particle numbers and take a closer look at the energy and its dependence on the number of electrons. It is known that this dependence qualitatively behaves as sketched in the following figure:
The exact xc functional connects the energies for integer particle numbers by straight lines and features derivative discontinuities $\Delta^{xc}$ at integer particle numbers. The local density approximation (LDA) on the other hand shows a smooth behavior.
Based on the equation for the fundamental band gap given above we can derive another expression for the exact xc functional: $$E_g = \lim_{\eta \rightarrow 0^+} \left\lbrace \left.\frac{\delta E[n]}{\delta n(\boldsymbol{r})}\right|_{N+\eta} - \left.\frac{\delta E[n]}{\delta n(\boldsymbol{r})}\right|_{N-\eta} \right\rbrace$$ where $n(\boldsymbol{r})$ is the density.
By plugging in Janak's theorem $\partial E / \partial n_i = \epsilon_i$ and the derivative discontinuity one ends up with $$E_g = \epsilon_{N+1} - \epsilon_{N} + \Delta_N^{xc}$$ where $\epsilon_i$ denotes the energy of i-th electron state in the Kohn-Sham system.
Detailed derivations of this result are provided in, e.g., J. P. Perdew, M. Levy: Physical Content of the Exact Kohn-Sham Orbital Energies: Band Gaps and Derivative Discontinuities, Phys. Rev. Lett. 51, 1884 (1983) or E. Engel, R. M. Dreizler: Density Functional Theory - An Advanced Course, Springer (2011).
The essence of this result is that even with the exact xc functional the Kohn-Sham band structure does not provide the fundamental band gap of the real interacting-electron system as it does not include the finite and positive derivative discontinuity.
Local and semilocal approximations to the xc functional like LDA or GGAs do not feature the discussed derivative discontinuities. But one can provide a simple hand-waving reason why the band structure underestimates the gap in this case.
One contribution to the energy of the Kohn-Sham system is the Hartree energy $$E_H[n] = \frac{1}{2} \int \frac{n(\boldsymbol{r}) n(\boldsymbol{r}')}{|\boldsymbol{r} - \boldsymbol{r}'|} d^3 r d^3 r'.$$ By considering a simple single-electron system like the hydrogen atom it is obvious that this energy contribution implies an unphysical self-interaction of the electron with itself.
This self-interaction has to be compensated by the xc energy but unfortunately an exact cancellation is not possible with local and semilocal xc functionals. A part of this unphysical energy contribution therefore remains and pushes the energies of the occupied states upwards. If a state is not occupied it does not contribute to the density and therefore there is no self-interaction for such states.
The band gap separates the occupied from the unoccupied states. Since the occupied states are lower in energy this implies a reduction of the gap.
DFT is based on two important theorems:
(1) Hohenberg & Kohn: the potential and the density are connected by a one-to-one map
(2) Kohn & Sham: there is always a non-interacting reference system (map: V_xc: non-interacting <-> interacting problem) having the same density as the interacting one.
In a nutshell: the potential and the density of the interacting system can be represented by a non-interacting potential / density.
So, DFT itself is exact in the ground state charge density if one knows the exact V_xc. Usually, V_xc is taken for a system where we have access to both solutions: the interacting and the non-interacting one. The most common reference system is the homogeneous (non-)interacting electron gas.
To your question: strictly speaking, transport properties are excitation properties. Thus engineer is correct in that point. The Kohn-Sham eigenvalues are the eigenspectrum of the non-interacting reference system and not the spectrum of the interacting problem (they might be totally different)! Surprisingly, it turned out that the Kohn-Sham spectrum is for many cases close to the excitation spectrum. The interpretation, however, as an excitation spectrum is mathematically not justified. It is only valid for Hartree-Fock (see Koopman's theorem). So the whole business of "predicting" band gaps within DFT(optimized V_xc) is emperically founded.
A comment to PuZhang: of course, one can improve V_xc's, but in order to interpret the Kohn-Sham eigenstates as excitations, and thus to derive "band gaps", one has to proceed in a different way. During the derivation of the Kohn-Sham equations, one can add a constraint forcing the eigenvalue spectra to be identical between the interacting and non-interacting system. However, whether one is still capable of finding a suitable approximation to V_xc in that case is yet to be proven.
All the best, Marc
DFT is exact concerning ground state properties. However, the bandgap is not a ground state property.
Not sure, if this simple explanation is correct, but I find it somehow intuitive: in order to speak about a bandgap, you either need a (at least fictitious) electron in the conduction band, which therefore is in an excited state, or you need a perturbation, which would lift an electron up and therefore also is not within the ground state.
