Density functional theory: logical steps from Hohenberg-Kohn theorems to Kohn-Sham equations

Question

I'm trying to learn density functional theory (DFT), using Engel & Dreisler, Sholl & Steckel, and Wikipedia as 3 different sources of information. Although I have made some progress, I am struggling to grasp the overall structure of the logical argument. In particular, I am unable to work out the logical steps that link the Hohenberg-Kohn theorems to the Kohn-Sham equations.

Below, I've sketched my best attempt at formulating a possible logical link between the Hohenberg-Kohn theorems and the Kohn-Sham equations. I would be extremely grateful if you could tell me if the logic I've sketched is correct. If my logic is correct, I'd appreciate it if you could briefly summarise how I can fill in the missing details, or provide links to suitable references that supply these details. If my logic is incorrect, I'd appreciate it if you could point out where I have gone wrong. I should also mention that I'm much more concerned with the overall structure of the logic than with detailed proofs of individual steps.

---

Let $U(\mathbf r, \mathbf r')$ be a function representing the interaction potential between pairs of electrons. This $U(\mathbf r, \mathbf r')$ is fixed throughout our entire discussion.

First, I'll summarise my understanding of the Hohenberg-Kohn theorems.

Let $\mathcal V$ be the set of all possible external potential functions, which we denote by $V(\mathbf r)$.

For each $V(\mathbf r)$, there is a corresponding Schrodinger equation $$ \hat H_V \ \psi(\mathbf r_1, \dots, \mathbf r_N) = E \ \psi(\mathbf r_1, \dots, \mathbf r_N) \qquad \qquad (\star)$$ where $$ \hat H_V = - \frac{ \hbar^2}{2m} \sum_{i=1}^N \nabla_i^2 + \sum_{i=1}^N V(\mathbf r_i) + \sum_{i =1}^N \sum_{j=i+1}^N U(\mathbf r_i, \mathbf r_j) .$$

If $\psi_0(\mathbb r_1, \dots, \mathbf r_N)$ is the ground state solution of this Schrodinger equation (i.e. the solution corresponding to the lowest $E$), then $$ n_0(\mathbf r) = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \left( \sum_{i=1}^N \delta^3 (\mathbf r_i - \mathbf r) \right) \left| \psi_0(\mathbf r_1, \dots, \mathbf r_N) \right|^2$$ is the ground state charge density.

Let $\mathcal N$ be the set of all ground state charge density functions $n_0(\mathbf r)$ arising from some choice of external potential $V(\mathbf r)$.

Let $f : \mathcal V \to \mathcal N$ be the function that maps each $V(\mathbf r)$ to the corresponding $n_0(\mathbf r)$. Clearly, $f$ is surjective.

A non-trivial result that can be proved is:

First Hohenberg-Kohn theorem: The function $f : \mathcal V \to \mathcal N$ is also injective.

Because $f : \mathcal V \to \mathcal N$ is injective, the next definition makes sense.

Definition: The energy functional. For a given external potential function $V(\mathbf r)$, we define the energy functional $E_V : \mathcal N \to \mathbb R$ as follows.

Given a charge density function $n_0'(\mathbf r)$ in $\mathcal N$, let $V'(\mathbf r)$ be the external potential function that is mapped by $f$ to $n_0'(\mathbf r)$. (By the first Hohenbert-Kohn theorem, this $V'(\mathbf r)$ is unique.) Let $\psi_0'(\mathbf r_1, \dots, \mathbf r_N)$ be the ground state solution of the Schrodinger equation $\hat H_{V'} \psi' = E' \psi'$. (So $n'_0(\mathbf r)$ is the charge density function associated with $\psi_0'$.)

We define the image of $n_0'(\mathbf r)$ under the function $E_V : \mathcal N \to \mathbb R$ to be $$ E_V [ n_0' ] = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \bar{\psi}'_0(\mathbf r_1, \dots, \mathbf r_N) \ \hat H_V \ \psi'_0(\mathbf r_1, \dots, \mathbf r_N) .$$ This is defines the functional $E_V : \mathcal N \to \mathbb R$.

We then have the following result, which is also non-trivial:

Second Hohenberg-Kohn theorem: For any $V(\mathbf r)$ in $\mathcal V$, let $n_0(\mathbf r)$ be the image of $V(\mathbf r)$ under the function $f : \mathcal V \to \mathcal N$.

Then $$ n_0(\mathbf r) = \text{argmin}_{n_0'(\mathbf r) \in \mathcal N} \ E_V[n_0'].$$

The Hohenberg-Kohn theorems are interesting from a theoretical perspective. However, they are useless in practice, since, without an explicit expression for the functional $E_V : \mathcal N \to \mathbb R$, it is impossible to solve the above variational problem to determine the $n_0(\mathbf r)$ corresponding to a particular $V(\mathbf r)$.

As I understand it, the Kohn-Sham equations provide a concrete expression for $E_V : \mathcal N \to \mathbb R$, making it possible to solve this variational problem.

To go from the Hohenberg-Kohn theorems to the Kohn-Sham equations, we need to convert the variational problem in the second Hohenberg-Kohn theorem into a variational problem involving the energy functional associated with the Kohn-Sham equations. My best guess is that we need a theorem along the lines of what I've written below.

Possible theorem. There exists an exchange-correlation energy functional $\widetilde E^{\text{xc}} : \mathcal N \to \mathbb R$ such that, for any $V(\mathbf r)$ in $\mathcal V$, there exists set of wavefunctions $\mathcal W_V$ such that:

A: For any wavefunction $\psi(\mathbf r_1, \dots, \mathbf r_N)$ in $\mathcal W_V$, the corresponding charge density function $$ n_0(\mathbf r) = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \left( \sum_{i=1}^N \delta^3 (\mathbf r_i - \mathbf r) \right) \left| \psi(\mathbf r_1, \dots, \mathbf r_N) \right|^2$$ is contained in $\mathcal N$.

Thus it makes sense to talk about a function $g_V : \mathcal W_V \to \mathcal N$ that maps each wavefunction $\psi(\mathbf r_1, \dots, \mathbf r_N)$ in $\mathcal W$ to its corresponding charge density function $n_0(\mathbf r)$.

B: The function $g_V : \mathcal W_V \to \mathcal N$ is surjective.

C: The function $g_V : \mathcal W_V \to \mathcal N$ is also injective.

D: For any $n_0(\mathbf r)$ in $\mathcal N$, if $\psi(\mathbf r_1, \dots, \mathbf r_N)$ is the (unique) wavefunction in $\mathcal W_V$ that is mapped by $g_V$ to $n_0(\mathbf r)$, then $$E_V[n_0] = E_{V}^{\text{KS}} [\psi] ,$$ where $$ E_{V}^{\text{KS}} [\psi] = E^{\text{kin}}[\psi] + E^{\text{ext}}[\psi] + E^{\text{HF}}[\psi] + E^{\text{xc}}[\psi],$$ with $$ E^{\text{kin}}[\psi] = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \bar{\psi}(\mathbf r_1, \dots, \mathbf r_N) \left( - \frac{ \hbar^2}{2m} \sum_{i=1}^N \nabla_i^2 \right) \psi(\mathbf r_1, \dots, \mathbf r_N),$$ $$ E^{\text{ext}}[\psi] = \left. \left( \int d^3 \mathbf r \ V(\mathbf r) n(\mathbf r) \right) \right|_{n(\mathbf r) = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \left( \sum_{i=1}^N \delta^3 (\mathbf r_i - \mathbf r) \right) \left| \psi(\mathbf r_1, \dots, \mathbf r_N) \right|^2}$$ $$ E^{\text{HF}}[\psi] = \left. \left( \frac 1 2 \int d^3 \mathbf r d^3 \mathbf r' \ U(\mathbf r, \mathbf r') n(\mathbf r) n(\mathbf r') \right) \right|_{n(\mathbf r) = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \left( \sum_{i=1}^N \delta^3 (\mathbf r_i - \mathbf r) \right) \left| \psi(\mathbf r_1, \dots, \mathbf r_N) \right|^2}$$ $$ E^{\text{xc}}[\psi] = \left. \widetilde E^{\text{xc}}[n] \right|_{n(\mathbf r) = \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \left( \sum_{i=1}^N \delta^3 (\mathbf r_i - \mathbf r) \right) \left| \psi(\mathbf r_1, \dots, \mathbf r_N) \right|^2}. $$

Thus, solving the variational problem in the second Hohenberg-Kohn theorem is equivalent to solving a (hopefully easier) variational problem on $\mathcal W_V$:

Corollary of possible theorem: For any $V(\mathbf r)$ in $\mathcal V$, let $\psi(\mathbf r_1, \dots, \mathbf r_N)$ be the solution to the variational problem $$ \psi(\mathbf r_1, \dots, \mathbf r_N) = \text{argmin}_{\psi'(\mathbf r_1, \dots, \mathbf r_N) \in \mathcal W_V} E_V^{\text{KS}} [\psi'].$$

If $n_0(\mathbf r)$ is the image of $\psi(\mathbf r_1, \dots, \mathbf r_N)$ under the function $g_V: \mathcal W_V \to \mathcal N$, then $$n_0(\mathbf r) = \text{argmin}_{n'_0(\mathbf r) \in \mathcal N} \ E_V[n'_0].$$ Furthermore, $$E_V^{KS} [\psi] = E_V[n_0].$$

I've never seen this "possible theorem" written down anywhere. This so-called theorem is merely something I've concocted to connect the logical dots. I don't know whether this so-called theorem is correct, let alone how one might go about proving it.

First of all, I haven't seen any mention of the set $\mathcal W_V$ written down in any of the sources that I'm studying from. This worries me.

My condition B is similar to a concept that Engel & Dreisler refer to as "non-interacting $V$-representability". I didn't understand Engel & Dreisler's discussion about non-interacting $V$-representability. The impression I took away from the book is that non-interacting $V$-representability cannot be made to hold in general - and therefore, my condition B cannot be made to hold in general either. However, if we pretend that my condition B does hold and proceed with the rest of the calculation, then the final answers we will get for the ground state charge density will turn out to be approximately correct. I'm not sure if I have interpreted the book correctly, and I have no idea about how to formulate this statement in a precise manner.

My condition D is fairly non-controversial. It is implicit from the practical calculational method for calculating ground state charge densities using DFT that is outlined by all four of my sources.

I have never seen conditions A and C written down anywhere. However, if A were not true, then the codomain of the function $g_V$ would need to be larger than $\mathcal N$, so the variational problem on $\mathcal W_V$ might not be equivalent to the variational problem on $\mathcal N$ that appears in the second Hohenberg-Kohn theorem, which would make the whole thing illogical.

And if C were not true, then the equation $E_V[n_0] = E_{V}^{\text{KS}} [\psi]$ would be nonsensical, since there might exist multiple $\psi(\mathbf r_1, \dots, \mathbf r_N)$'s in $\mathcal W_V$ that are mapped by $g_V$ to a given $n_0(\mathbf r)$ in $\mathcal N$.

Anyway, parking these controversies, we're still not quite home and dry. To complete the logic, we need a theorem along the lines of what I've written below, to show that our variational problem on $\mathcal W_V$ is equivalent to solving the Kohn-Sham equations.

Another possible theorem: Let $V(\mathbf r)$ be an external potential function in $\mathcal V$.

1. The solution to the variational problem $$ \psi(\mathbf r_1, \dots, \mathbf r_N) = \text{argmin}_{\psi'(\mathbf r_1, \dots, \mathbf r_N) \in \mathcal W_V} E_V^{KS} [\psi']$$ takes the form of a Slater determinant of single-particle wavefunctions, $$ \psi(\mathbf r_1, \dots, \mathbf r_N) = \frac{1}{\sqrt{N}} \det \begin{bmatrix} \chi_1(\mathbf r_1) & \dots & \chi_N(\mathbf r_1) \\ \vdots & \ddots & \vdots \\ \chi_1(\mathbf r_N) & \dots & \chi_N(\mathbf r_N) \end{bmatrix} .$$

2. These single-particle wavefunctions $\chi_i(\mathbf r)$ all satisfy the Kohn-Sham equation $$ \hat H^{KS}_V \chi_i(\mathbf r) = \epsilon_i \chi_i(\mathbf r),$$ where $$ \hat H^{KS}_V = - \frac{-\hbar^2}{2m} \nabla^2 + V(\mathbf r) + V^{\text{HF}}(\mathbf r) + V^{\text{xc}}(\mathbf r), $$ with $$ V^{\text{HF}}(\mathbf r) = \left. \int d^3 \mathbf r' U(\mathbf r, \mathbf r') n(\mathbf r') \right|_{n(\mathbf r') = \sum_{i = 1}^N \left| \chi_i(\mathbf r') \right|^2 },$$ $$ V^{\text{xc}}(\mathbf r) = \left. \frac{\delta E^{\text{xc}}}{\delta n}[n] \right|_{n(\mathbf r) = \sum_{i = 1}^N \left| \chi_i(\mathbf r) \right|^2 }.$$

3. In fact, these $\chi_i$'s are the solutions to the Kohn-Sham equation corresponding to the lowest $N$ eigenvalues $\epsilon_i$.

This "possible theorem", together with the previous "possible theorem", completes the logical link between the Hohenberg-Kohn theorems and the Kohn-Sham equations, and provides a practical way to compute the ground state charge density and ground state energy for a given external potential. (The only caveat is that we don't have an explicit expression for $E^{\text{xc}}[n]$, but I'm not worried about this right now since I understand that many researchers have proposed approximate expressions for $E^{\text{xc}}[n]$ that are well motivated and have a track record of producing accurate results.)

So how might we go about proving this so-called theorem?

It is tempting to say that statement 1 follows from statement 2, since statement 2 shows that we are solving a non-interacting Schrodinger equation. But this is circular logic! It is circular to argue that statement 1 follows from statement 2, while statement 2 in turn only makes sense in light of statement 1.

Also, assuming statement 1 is true, it's not completely trivial to derive statement 2 from statement 1. Sure, I can see that if you substitute the Slater determinant ansatz from statement 1 into the expression for $E_V^{\text{KS}}[\psi]$ and mindlessly write down the Euler-Lagrange equations for $\psi$'s that extremise $E_V^{\text{KS}}[\psi]$, then the Kohn-Sham equations in statement 2 will drop out. But I don't trust this calculation for two reasons.

• Firstly, I think that this calculation requires some additional Lagrange multipliers to enforce the constraint that the $\chi_i$'s must be orthonormal, thus complicating the Euler-Lagrange equations.
• Secondly, we really want to minimise $E_V^{\text{KS}}[\psi]$ over only those $\psi$'s that lie in $\mathcal W_V$, but without further knowledge of the geometry of $\mathcal V_W$, I can't tell if taking functional derivatives is a legitimate mathematical operation on $\mathcal W_V$. (To take a derivative of a function at a point on the space, that space must contain an "open neighbourhood" around the point, so that we can measure how the function changes when we perturb the point in various directions. It isn't clear if $\mathcal W_V$ satisfies this property.)

Finally, statement 3 isn't obvious. After all, $E_V^{\text{KS}}[\psi]$ is not equal to $\sum_{i=1}^N \epsilon_i$. So it's hard to see how one would go about proving that choosing the lowest $\epsilon_i$'s will give rise to the lowest possible $E_V^{\text{KS}}[\psi]$.

---

To summarise, I would appreciate it if you could tell me whether my two "possible theorems" (a) are correct, (b) are not strictly true but nonetheless give rise to final answers that are approximately correct, or (c) are completely off the mark.

If my possible theorems are correct or morally correct, then please be so kind as to sketch or provide links to arguments that will help me prove the claims that I am currently unable to prove.

If they are incorrect, then please tell me how to fix the errors.

My main priority right now is getting the overall structure of the logic clear in my mind. Understanding intricate proofs of individual claims is a lesser priority.

Answer 1

1. The Hohenberg-Kohn Theorems:

I think the first part of your question does indeed summarize well the Hohenberg-Kohn theorems. Apart from some minor details (such as that $V$ is unique up to an additive constant, or $\psi$ is unique up to a phase), I think I have nothing to add. Your explanation should very well agree with the ones given in the books of e.g. Dreizler.

2. $v$-Representability:

As you've pointed out already, the discussion basically takes places for a fixed interaction $U$ (say e.g. Coulomb interaction) and kinetic energy $T$ (and particle number $N$). For a fixed $U$, we are interested in all possible ground states (here we assume non-degeneracy, which is not much of a restriction for the following discussion) generated by the different external potentials $V$. Your set $\mathcal N$ is often called set of $v$-representable densities: It contains all densities which can be realized from the ground state $\psi_0$ for some external potential $V$.

Note that this is a relative notion: Given a density $n$ (what a density, or better a $N$-representable density is can be defined mathematically), it might be $v$-representable for $U$, but not for a different interaction $U^\prime$.

3. Non-interacting $v$-representability:

Of course,the Hohenberg-Kohn theorems also hold for $U=0$, i.e. non-interacting systems. Suppose the system you are interested in has for some external potential $V$ a ground state $\psi_0$ with ground state density $n_0$. This density is thus (interacting-)$v$-representable. It is additionally non-interacting $v$-representable if there is a non-interacting system $(U=0)$ for which there is an external potential $V_\mathrm{eff}$ such that the ground state $\phi_0$ gives rise to $n_0$.

Thus, if you have a density which is both $v$-representable and non-interacting $v$-representable, there are (at least) two systems which have a ground state giving the said density. Whether or not every $v$-representable density is also non-interacting $v$-representable is, as far as I know, an unsettled problem.

4. Kohn-Sham Systems:

Assuming for now that indeed every $v$-representable density is also non-interacting $v$-representable, we can discuss the Kohn-Sham systems. Suppose you knew the effective potential $V_\mathrm{eff}$ which gives rise to the ground state density $n_0$ of interest. Then you could take this potential, solve the non-interacting Schrödinger equation and then simply extract from the ground state $\phi_0$ the ground state density $n_0$, which you then may plug into the (approximated) energy functional.

However, the conditional statement is important. We know that there exists some $V_\mathrm{eff}$ with the desired properties, but we don't know a priori how it looks like. There are ways however, explained in basically any reference I gave above, to construct this potential. What you will find is that $V_\mathrm{eff}$ depends on the functional derivative of the energy functional and in particular it depends on the ground state density.

This then leads to the known fact that you have to solve the Kohn-Sham equations self-consistently. In other words, given some approximation for the energy functional, you may start with an initial guess for the ground state density $\tilde n_0$, solve the Kohn-Sham equations and get a new ground state density. Repeat the cycle until convergence (which might be a topic on its own).

---

Answer to questions in the comments:

The external potential operator for your system of $N$ (identical) fermions (e.g. electrons) is of the form $V=\sum\limits_{i=1}^N v(x_i)$. The function $v:\mathbb R^3\longrightarrow \mathbb R$ is the external potential, which every electron feels (e.g. the presence of the fixed ions of a lattice in some solid material). The effective potential also takes this form $V_\mathrm{eff} =\sum\limits_{i=1}^N v_{\mathrm {eff}}(x_i)$. The Hamiltonian of the non-interacting system is by definition $$H_{\mathrm{eff}}=T+V_{\mathrm{eff}}= \sum\limits_{i=1}^N h_i \quad ,$$ where $h_i =- \frac{\hbar^2}{2m}\nabla_i^2 + v_{\mathrm {eff}}(x_i)$. Without going into too much detail, we can show (with some assumptions, e.g. non-degeneracy) that the ground state is of the form of a Slater-determinant: If $$h\varphi_j=\epsilon_j \varphi_j \quad ,$$ then an eigenstate of $H_{\mathrm{eff}}$ is given by $\psi\sim \varphi_{j_1} \wedge \varphi_{j_2} \wedge \ldots \wedge \varphi_{j_N}$ with energy $E=\sum\limits_{k=1}^N \epsilon_{j_k}$. This also shows you that the ground state of this system is the one such that its corresponding energy is build from the $N$ lowest $\epsilon_j$'s.

Consequently, in order to solve the non-interacting Schrödinger equation with the Hamiltonian $H_\mathrm{eff}$, you can solve the single-particle Schrödinger equation from which you take the solutions $\varphi_j$ with the $N$ lowest energies $\epsilon_j$ to build your ground state Slater determinant, from which in turn you get the ground state density. But $v_\mathrm{eff}$ depends on the ground state density $n_0$, which we have to determine. This makes the corresponding single-particle Schrödinger equation a non-linear problem and hence we have to solve the Kohn-Sham equations self-consistently.

Answer 2

I wanted to point out that ChatGPT should be given zero weight, but then I realised that your problem is far worse than that.

You are dropping into the hole that we are deliberately leaving around. The fact of the matter is that there is no link between HK and KS, at least not yet, and the issue is precisely as you stated: We have no way to write down the functional that using HK would need us to have.

However, HK is still necessary, and performs a rather dramatically practical rôle in physics, which is to quell disputes. The other day I was just listening to the people who did TDDFT, and they proved a version of HK for TDDFT so that they can point out that the calculations necessarily has to converge to a consistent picture between the different implementations and choices. For an example of its use, we all know that DFT is notoriously bad at finding the correct band gap of materials, but as long as you are all using PBE, and the same lattice structures, then it does not matter which pseudopotential library or program you use, you ought to get the exact same band gap, wrong as that band gap may be. You can simply point to a discrepancy in the band gap from what it ought to be, and assert that someone did not pick conditions well enough for their simulation to converge to the correct results. This is contrasted with, say, the charge distribution, which is not protected by HK, and so there are many possible ways to get slightly different variations of this.

Between HK and KS is a bait-and-switch. We only care about KS because it is practically useful to have cheap approximate answers, not that it is theoretically great. We are still solving Schrödinger-like equations, so, while it is nonsense, it is not completely nonsense. Nobody is claiming that you should take this as gospel.

---

You are also making mistakes in the understanding of the exchange correlation.

First, a nitpick, your $U(\vec r,\vec r\,^\prime)$ is really just the Coulomb potential between electrons. Note that we are ignoring all the QED corrections to this. Since the transition to KS is such a violent act, this ignoring should be more than good enough.

More importantly, when you write down KS equations, you do not do Slater determinants.

It is important to understand the origins of the exchange and correlation terms. When you do Hartree Fock, you know that you have to use at least a Slater determinant to enforce fermion indistinguishability, and that gives rise to Pauli's Exclusion Principle. The exchange term actually comes from Coulomb potential interacting with this Slater determinant. i.e. you ought to consider exchange term as part of Coulomb potential, not something separate.

However, Hartree Fock is itself an approximation. Slater determinants are an approximation. When you write down a Slater determinant, the probability of finding an electron at position $\vec r$ is independent of the probability of finding an electron at another position $\vec r\,^\prime$, and that is just plain nonsense. In reality, electrons are going to be correlated in their positions, and so we define any part of the actual energy computation of a quantum system that Hartree Fock cannot capture, as the correlation energy term.

In particular, consider a pair of electrons, where $\chi$ labels the spin part, then the fermion symmetry lies in $$\Psi(\vec r_2,\chi_2,\vec r_1,\chi_1)=-\Psi(\vec r_1,\chi_1,\vec r_2,\chi_2)\tag1$$ A Slater determinant allows us to get this, \begin{align}\tag2 \Psi&\approx\frac1{\sqrt2} \begin{vmatrix} \psi_1(\vec r_1)\chi_1 & \psi_1(\vec r_2)\chi_1 \\ \psi_2(\vec r_1)\chi_2 & \psi_2(\vec r_2)\chi_2 \end {vmatrix}=\frac1{\sqrt2}[ \psi_1(\vec r_1)\chi_1\psi_2(\vec r_2)\chi_2 - \psi_2(\vec r_1)\chi_2\psi_1(\vec r_2)\chi_1 ] \end{align} But the true wavefunction can only be represented as $$\tag3\Psi=\psi_S\left(\frac{\vec r_1+\vec r_2}2\right)\psi_A(\vec r_2-\vec r_1,\chi_1,\chi_2)\qquad,\qquad\psi_A(-\vec r,\chi_2,\chi_1)=-\psi_A(\vec r,\chi_1,\chi_2)$$ where only either the spatial or the spin part gets the anti-symmetry (for only two electrons—if there are more, the actual symmetry is horrendous).

---

One might be tempted to consider Hartree Fock as the thing to approximate towards, ignoring the correlation. But this is actually very wrong. When we have delocalised electrons, say in a metal, or even the bonding electrons in an insulator, it is known that the Hartree Fock method would give a band gap that is way too big. This means that it is natural to think of exchange and correlation to be somewhat related in a quantum chemistry computation, and we should thus treat them together. Since the exchange term comes from Slater determinants, to make its effect smaller, it is therefore sensible to not do Slater determinants in DFT in general. Instead, the effect of exchange and correlation goes into each particular type of functional used. It is also very helpful, because the taking of Slater determinant is considered rather expensive.

Answer 3

Chapter 5 of Robert van Leeuwen's notes provides the missing links that I was looking for. This is one of references that was kindly recommended by Tobias in a comment.

For the benefit of future readers, I'll summarise what I've learnt in my own words. If there are mistakes in what I write, then please do leave a comment, as this will help me learn.

First, we have the following definitions, which were beautifully described in Tobias' answer.

Definition: A charge density function $n(\mathbf r)$ is $V$-representable if it is the charge density function associated with the ground state wavefunction for the Hamiltonian $$\hat H_V = - \frac{\hbar^2}{2m} \sum_{i=1}^N \nabla_i^2 + \sum_{i=1}^N V(\mathbf r_i) + \sum_{i=1}^N \sum_{j=i+1}^N U(\mathbf r_i, \mathbf r_j), $$ for some choice of external potential $V(\mathbf r)$.

If $n(\mathbf r)$ is $V$-representable, then we use the notation $\psi[n](\mathbf r_1, \dots, \mathbf r_N)$ and $V[n](\mathbf r)$ to denote the associated wavefunction and external potential. $\psi[n]$ and $V[n]$ are unique for a given $n$, by the first Hohenberg-Kohn theorem.

Similarly, a charge density function $n(\mathbf r)$ is non-interacting $V$-representable if it is the charge density function associated with the ground state wavefunction for the non-interacting Hamiltonian $$ \hat H^{\text{KS}}_{V^{\text{KS}}} = - \frac{\hbar^2}{2m} \sum_{i=1}^N \nabla_i^2 + \sum_{i=1}^N V^{\text{KS}}(\mathbf r_i), $$ for some choice of external potential $V^{\text{KS}}(\mathbf r)$.

If $n(\mathbf r)$ is non-interacting $V$-representable, then we use $\psi^{\text{KS}}[n](\mathbf r_1, \dots, \mathbf r_N)$ and $V^{\text{KS}}[n](\mathbf r)$ to denote the associated wavefunction and external potential. These, too, are unique for a given $n$.

Now, we take a leap of faith.

Leap of faith: Every $V$-representable charge density function is also non-interacting $V$-representable.

[Actually, I'm not entirely sure if this statement should be viewed as a leap of faith. It could be that there are arguments showing that even if this statement isn't quite true, it is "approximately" true in such a way that if you blissfully proceed with your calculations, you'll end up with final answers that are approximately correct. I need to read up on this...]

Next, we write down the definition of the exchange-correlation energy functional.

Definition: The exchange-correlation energy $E^{\text{xc}}[n]$ of a $V$-representable charge density function $n(\mathbf r)$ is defined by the equation $$ E^{\text{xc}}[n] = E[n] - E^{\text{kin}}[\psi^{KS}[n]] - E^{\text{ext}}[n] - E^{\text{H}}[n]. \qquad \qquad (\star)$$

Here, $E[n]$ is defined to the ground state energy of the wavefunction $\psi[n]$, with respect to the (interacting) Hamiltonian $\hat H_{V[n]}$.

The other terms are defined by $$ E^{\text{kin}}[\psi^{\text{KS}}[n]] = - \frac{ \hbar^2}{2m} \sum_{i=1}^N \int d^3 \mathbf r_1 \dots d^3 \mathbf r_N \ \bar{\psi}^{\text{KS}}[n] \nabla_i^2 \psi^{\text{KS}}[n] ,$$ $$ E^{\text{ext}}[n] = \int d^3 \mathbf r \ V[n](\mathbf r) n(\mathbf r)$$ $$ E^{\text{H}}[n] = \frac 1 2 \int d^3 \mathbf r d^3 \mathbf r' \ U(\mathbf r, \mathbf r') n(\mathbf r) n(\mathbf r').$$ Note that in order to define $E^{\text{kin}}[\psi^{\text{KS}}[n]]$, we need $\psi^{\text{KS}}[n]$ to exist, and for that, we're relying on our "leap of faith".

[In my original post, I got into a tangle trying to prove the existence of a functional $E^{\text{xc}}[n]$ satisfying certain properties. Here, the logic is much cleaner: since I'm providing the definition of a functional $E^{\text{xc}}[n]$, there is nothing to prove!]

There is a simple relationship between the potentials in the interacting and non-interacting theories that give rise to the charge density $n$.

Theorem: Let $n(\mathbf r)$ be a $V$-representable charge density, which is also non-interacting $V$-representable, by our "leap of faith". Then $$ V^{\text{KS}}[n](\mathbf r) = V[n](\mathbf r) + V^{\text{H}}[n](\mathbf r) + V^{\text{xc}}[n](\mathbf r),$$ with $$ V^{\text{H}}[n](\mathbf r) = \int d^3 \mathbf r' U(\mathbf r, \mathbf r') n(\mathbf r'),$$ $$ V^{\text{xc}}[n](\mathbf r) = \frac{\delta E^{\text{xc}}}{\delta n}[n](\mathbf r).$$

The proof of this theorem is surprisingly simple. It's just a matter of taking functional derivatives of both sides of equation $(\star)$ with respect to $n$. (In one step, you have to appeal to the fact that $\psi[n]$ and $\psi^{\text{KS}}[n]$ are unit-normalised eigenfunctions of $\hat H_{V[n]}$ and $\hat H^{\text{KS}}_{V^{\text{KS}}[n]}$ in order to get nice expressions for $E^{\text{ext}}[n] - E[n]$ and $-E^{\text{kin}}[\psi^{\text{KS}}[n]]$. But that's about as hard as it gets.) The proof is written out in Chapter 5 of van Leeuwen's notes, which I have linked.

Finally, as Tobias explains nicely in his answer, $\psi^{\text{KS}}[n]$ is by definition the ground state wavefunction of the non-interacting Hamiltonian $\hat H^{KS}_{V^{\text{KS}}[n]}$. Therefore, $\psi^{\text{KS}}[n]$ can be expressed in terms of Slater determinants.

Theorem: $\psi^{\text{KS}}[n]$ is given by the Slater determinant $$ \psi^{\text{KS}}[n](\mathbf r_1, \dots, \mathbf r_N) = \frac{1}{\sqrt{N}} \det \begin{bmatrix} \chi_1(\mathbf r_1) & \dots & \chi_N(\mathbf r_1) \\ \vdots & \ddots & \vdots \\ \chi_1(\mathbf r_N) & \dots & \chi_N(\mathbf r_N) \end{bmatrix},$$

where the $\chi_i(\mathbf r)$'s are eigenstates of the Kohn-Sham equation $$ \left(- \frac{\hbar^2}{2m} \nabla^2 + V^{\text{KS}}[n](\mathbf r) \right) \chi_i(\mathbf r) = \epsilon_i \chi_i(\mathbf r),$$ corresponding to the $N$ lowest eigenvalues $\epsilon_i$.

[In my original post, $\psi^{KS}[n]$ was defined to be the solution to a variational problem, which led to a lot of confusion about whether $\psi^{KS}[n]$ could be expressed as a Slater determinant. Here, $\psi^{KS}[n]$ is explicitly defined to be the ground state wavefunction of a non-interacting theory, so it's obvious that it can be expressed as a Slater determinant. And as a side note, it's interesting that the second Hohenberg-Kohn theorem isn't used at all.]

And that's it! Given an external potential $V$, the procedure is:

1. Use the Kohn-Sham equation to find the ground state charge density $n$. This is done iteratively, until convergence:
   • Using our current expression for $n$, solve the Kohn-Sham equation to obtain an improved expression for $\psi^{\text{KS}}[n]$. (This is where our choice of $V$ plays a role: in the Kohn-Sham equation, we set $V[n]$ equal to $V$.)
   • Using our current expression for $\psi^{\text{KS}}[n]$, compute an improved expression for $n$.
2. Plug our final answers for $n$ and $\psi^{\text{KS}}[n]$ into our expression for $E[n]$ to get the ground state energy.

This all makes sense now - the logic is nice and clear. I'm extremely grateful to Tobias for being so patient in helping me.