The variational method can be used to calculate the ground state wavefunction of a quantum many-body system. Suppose we have a trial wavefunction representing the ground state of the system, parameterised by a set of parameters $\boldsymbol{\alpha}$, which can be expressed in the finite basis $|\sigma\rangle$, $$ |\psi(\boldsymbol{\alpha})\rangle=\sum_{\sigma} \langle\sigma|\psi(\boldsymbol{\alpha})\rangle |\sigma\rangle. $$ One can iteratively improve the Ansatz $|\psi(\boldsymbol{\alpha})\rangle$ by adjusting the parameters $\boldsymbol{\alpha}$ so that the expectation value of the many-body Hamiltonian attains a minimum. This can be done by setting $$ \boldsymbol{\alpha} \leftarrow \boldsymbol{\alpha} - \delta\boldsymbol{f}, $$ where $\delta$ is a small number and the force gradient is calculated as $$ f_i = \langle\Delta_i H\rangle-\langle\Delta_i \rangle\langle H\rangle, $$ where $\Delta_i$ is a diagonal operator. In the variational Monte Carlo method, the expectation values in the equation above are calculated approximately using a Markov Chain Monte Carlo sampling procedure, i.e. calculating $$ \langle \mathcal{O} \rangle = \frac{\langle \psi|\mathcal{O}| \psi\rangle}{\langle\psi|\psi\rangle}\approx \frac{ \sum_{\sigma\in S} |\langle \psi | \sigma \rangle |^2 \frac{\langle \sigma|\mathcal{O}| \psi\rangle}{\langle\sigma|\psi\rangle} }{\sum_{\sigma\in S} |\langle \psi | \sigma \rangle |^2} $$ for a subset $S$ of basis states. I don't understand the benefit of calculating the expectation values this way. Given that the matrix expressions for the operators $\Delta_i$ and $H$ are known, as is the expression of the trial wavefunction $|\psi\rangle$, the expectation values in the expression for $f_i$ above can be calculated directly, i.e. multiplying together a row vector, a matrix, and a column vector. Applying the Monte Carlo approach would require the calculation of similar expressions $\langle \sigma|\mathcal{O}| \psi\rangle$ multiple times for different $\langle \sigma|$, so it strikes me as being less efficient than a direct calculation.
1 Answer
When the number of finite states is such that you can simply calculate the expectation value by summing as you say, then no one uses Monte Carlo for that problem. The usual case when Monte Carlo is used is when the number of states is extremely large so doing the full sum is not feasible. For example the first major quantum Monte Carlo calculation was McMillan's variational Monte Carlo calculation of the ground state of liquid helium W. L. McMillan, Phys. Rev. 138 A442 (1965) https://journals.aps.org/pr/abstract/10.1103/PhysRev.138.A442 where the basis was the positions of up to 108 particles in a periodic simulation box. Even if you have a finite grid of say $1000$ points in the cube, this requires $1000^{108}$ basis states for 108 particles. That number is bigger than the number of electrons in the visible universe so it is not computationally feasible on a classical computer. Monte Carlo could sample the particle positions and give a reasonable answer even with computers available more than 50 years ago.
