I find the wording a little sloppy, because as written it seems to suggest somehow $\phi=\psi$, or that there is no degeneracy in the eigenspaces. This is false in general. However, the point is that if for a Hermitian operator you know all the critical points of the one-variable functional $\Lambda_1$, then you know all the critical points of the two-variable functional $\Lambda$, because you simply take pairs $(\phi,\psi)$ where $\phi,\psi$ are critical for $\Lambda_1$ and lie on the same level set (i.e $\Lambda_1(\phi)=\Lambda_1(\psi)$). See below for the notation and proof.
1. Critical points of the two variable function.
Let $E=\{(\phi,\psi)\in\mathcal{H}\times\mathcal{H}\,:\, \langle\phi|\psi\rangle\neq 0\}$, and consider the functional $\Lambda:E\to\Bbb{C}$ defined as
\begin{align}
\Lambda(\phi,\psi)&:=\frac{\langle\phi|H\psi\rangle}{\langle \phi|\psi\rangle}.
\end{align}
Now, at each point $(\phi,\psi)\in E$, and each $(\alpha,\beta)\in\mathcal{H}\times\mathcal{H}$, we can compute the directional derivative of $\Lambda$ at the point $(\phi,\psi)$ along the direction $(\alpha,\beta)$. This gives after a simple calculation:
\begin{align}
(D_{(\alpha,\beta)}\Lambda)(\phi,\psi)&:=
\frac{d}{dt}\bigg|_{t=0}\Lambda\left((\phi,\psi)+t(\alpha,\beta)\right)\\
&=\frac{d}{dt}\bigg|_{t=0}\Lambda\left(\phi+t\alpha,\psi+t\beta\right)\\
&=\frac{d}{dt}\bigg|_{t=0}\frac{\left\langle\phi+t\alpha|H\psi+ tH\beta\right\rangle}{\langle \phi+t\alpha|\psi+t\beta\rangle}\\
&=\frac{\langle\phi|\psi\rangle\left[\langle\alpha|H\psi\rangle+\langle\phi|H\beta\rangle\right]-\langle\phi|H\psi\rangle\left[\langle\alpha|\psi\rangle+\langle\phi|\beta\rangle\right]}{\langle\phi|\psi\rangle^2}\\
&=\frac{1}{\langle\phi|\psi\rangle^3}\left[\bigg\langle\alpha\bigg|H\psi-\frac{\langle\phi|H\psi\rangle}{\langle\phi|\psi\rangle}\psi\bigg\rangle
+\bigg\langle H^{\dagger}\phi+\overline{\frac{\langle\phi|H\psi\rangle}{\langle\phi|\psi\rangle}}\phi\bigg|\beta\bigg\rangle \right]\\
&=\frac{1}{\langle\phi|\psi\rangle^3}\left[
\bigg\langle\alpha\bigg| H\psi-\Lambda(\phi,\psi)\psi\bigg\rangle+
\bigg\langle H^{\dagger}\phi-\overline{\Lambda(\phi,\psi)}\phi\bigg|\beta\bigg\rangle
\right].
\end{align}
I’ve simply used the quotient rule, and that the inner product is bilinear (and continuous) so the ‘product rule’ applies to it, and finally I cleaned up the algebra and used the definition of $\Lambda$. This expression vanishes if and only if the stuff in square brackets vanishes. So, the directional derivative vanishes for all $(\alpha,\beta)$ if and only if we have the two equations
\begin{align}
H\psi-\Lambda(\phi,\psi)\psi&=0,\quad\text{and}\quad H^{\dagger}\phi-\overline{\Lambda(\phi,\psi)}\phi=0.\tag{$*$}
\end{align}
But now notice that we can express the eigenvalues slightly differently as well. For the first equation, “apply the bra $\langle\psi|$”, to get
\begin{align}
0&=\langle\psi|H\psi-\Lambda(\phi,\psi)\psi\rangle=\langle\psi|H\psi\rangle-\Lambda(\phi,\psi)\langle\psi|\psi\rangle.
\end{align}
Rearranging gives $\Lambda(\phi,\psi)=\Lambda(\psi,\psi)$. Likewise, from the second equation in $(*)$, we deduce that $\Lambda(\phi,\psi)=\Lambda(\phi,\phi)$.
Thus, a point $(\phi,\psi)$ is a critical point for the functional $\Lambda$ if and only if $(*)$ holds. In this case, we necessarily have $\Lambda(\phi,\psi)=\Lambda(\psi,\psi)=\Lambda(\phi,\phi)$.
2. Critical points of the one-variable function for Hermitian $H$.
Define $\Lambda_1:\mathcal{H}\setminus\{0\}\to\Bbb{R}$ as $\Lambda(\psi,\psi)$. Note that because $H$ is Hermitian, $\Lambda_1$ is real-valued:
\begin{align}
\Lambda_1(\psi)&=\Lambda(\psi,\psi)=\frac{\langle\psi|H\psi\rangle}{\|\psi\|^2}=\frac{\langle H^{\dagger}\psi|\psi\rangle}{\|\psi\|^2}=\frac{\langle H\psi|\psi\rangle}{\|\psi\|^2}=\overline{\frac{\langle\psi|H\psi\rangle}{\|\psi\|^2}}=\overline{\Lambda_1(\psi)}.
\end{align}
Now, what are the critical points of $\Lambda_1$? Well, a point $\psi\in\mathcal{H}\setminus\{0\}$ is critical if and only if for all $\alpha\in\mathcal{H}$, we have $D_{\alpha}(\Lambda_1)(\psi)=0$. By the chain rule, this directional derivative is simply $(D_{(\alpha,\alpha)}\Lambda)(\psi,\psi)$. So, using our formula from above, we have
\begin{align}
D_{\alpha}(\Lambda_1)(\psi)&=
(D_{(\alpha,\alpha)}\Lambda)(\psi,\psi)\\
&=
\frac{1}{\langle\psi|\psi\rangle^3}\left[
\bigg\langle\alpha\bigg| H\psi-\Lambda_1(\psi)\psi\bigg\rangle+
\bigg\langle H\psi-\Lambda_1\psi\bigg|\alpha\bigg\rangle
\right]\\
&=\frac{1}{\langle\psi|\psi\rangle^3}\cdot 2\text{Re} \bigg\langle\alpha\bigg| H\psi-\Lambda_1(\psi)\psi\bigg\rangle.
\end{align}
Note, I’ve used that $\Lambda_1$ is real-valued. Thus, this directional derivative vanishes for all $\alpha$ if and only if
\begin{align}
H\psi-\Lambda_1(\psi)\psi&=0.\tag{$**$}
\end{align}
Hence, $\psi$ is a critical point for $\Lambda_1$ if and only if $\psi$ is an eigenvector of $H$, in which case it has eigenvalue $\Lambda_1(\psi)$.
Putting it all together for Hermitian $H$.
From step 1, and the fact that for Hermitian $H$, the function $\Lambda$ with equal inputs (i.e $\Lambda_1$) is real valued, we see that a pair $(\phi,\psi)$ is a critical point for $\Lambda$ if and only if $\phi,\psi$ are eigenvectors of $H$ with the same (real) eigenvalue of $\Lambda(\psi,\psi)=\Lambda(\phi,\phi)=\Lambda(\phi,\psi)$. Of course you can write this eigenvalue as $\Lambda_1(\psi)=\Lambda_1(\phi)$.
From step 2 however, we observe that $\psi$ is a critical point for $\Lambda_1$ if and only if it is an eigenvector of $H$, in which case the eigenvalue equals $\Lambda_1(\psi)$.
In short, the critical points of $\Lambda_1$ are exactly the the eigenvectors. Conversely, for any two critical points $\phi,\psi$ of $\Lambda_1$ which lie on the same level set (i.e equal eigenvalues $\Lambda_1(\phi)=\Lambda_1(\psi)$), it follows that the pair $(\phi,\psi)$ is critical for $\Lambda$. Therefore, restricting yourself to the study of $\Lambda_1$ is no loss of generality, and you’re not ‘missing out’ on anything.
Extra good to know facts.
Theorem.
Let $H$ be a Hermitian operator on a Hilbert space $\mathcal{H}$. Then,
- all its eigenvalues are real
- distinct eigenspaces are orthogonal.
To prove the first part, let $\lambda$ be an eigenvalue. This means there is a non-zero vector $\psi$ such that $H\psi=\lambda\psi$. Hence,
\begin{align}
\lambda\langle\psi|\psi\rangle=\langle\psi|\lambda\psi\rangle=
\langle\psi|H\psi\rangle\underbrace{=}_{H=H^{\dagger}}\langle H\psi|\psi\rangle=\langle\lambda\psi|\psi\rangle=\overline{\lambda}\langle\psi|\psi \rangle.
\end{align}
Since $\psi$ is non-zero, $\langle\psi|\psi\rangle=\|\psi\|^2>0$, so we can divide it on both sides to conclude $\lambda=\overline{\lambda}$, and hence shows all eigenvalues are real.
For the second part, suppose $\lambda,\mu$ are distinct eigenvalues of $H$, and let $\psi,\phi$ be any eigenvectors with these respective eigenvalues. Then,
\begin{align}
\lambda\langle\phi|\psi\rangle=\langle\phi|\lambda\psi\rangle=\langle\phi|H\psi\rangle=\langle H\phi|\psi\rangle=\langle \mu\phi|\psi\rangle=\mu\langle\phi|\psi\rangle.
\end{align}
I used that $H=H^{\dagger}$ in the third equality and that $\mu$ is real in the last equality. Since $\lambda\neq \mu$, it follows that $\langle\phi|\psi\rangle=0$, thus showing the different eigenspaces are orthogonal.
