Self-consistency of Effective Hamiltonian equation One way to write an effective Hamiltonian for a Hamiltonian $H$ is via the equation
$$
\frac{1}{E-H_{eff}} = P\frac{1}{E-H}P
$$
where P projects into the subspace we're interested in, and $H_{eff}$ is the effective Hamiltonian of this subspace. This equation is meant to hold for all $E$. I've heard two justifications for this idea; the first is that we want to match the eigenvalues of $H$ and $H_eff$, and both the left and the right hand sides of the above equation have singularities at the eigenvalues of $H$. The second is that Taylor expanding this equation gives
$$
\sum_i \frac{H_{eff}^i}{E^i}=\sum_i\frac{PH^iP}{E^i}
$$
which, by matching powers of $E^i$, gives $H_{eff}^i=PH^iP$, so all powers of the Hamiltonians agree on the subspace.
In both of these justifications, it's not clear to me why we expect to be able to find an $H_{eff}$ that satisfies the equation. For example, if our subspace has only 2 basis elements, then $H_{eff}$ has four matrix elements to determine. In the first formulation, though, there are a potentially huge number of energies $E$ which we have to have poles at, and in the second formulation there are an infinite number of equations we need $H_{eff}$ to satisfy. Yet, we only have four unknowns to play with!
So, why do we think an $H_{eff}$ will exist? I would also be interested in better/more clear explanations of the effective Hamiltonian equation, since they might offer a better intuition for why we expect this equation to work.
 A: To my knowledge effective Hamiltonians based on projection operators were first introduced by Feshbach (Feshbach1958, Feshbach1962, Feshbach1968). The works by Fano (Fano1961) on resonances in Helium and Bloch (notoriously hard to find reference, because the paper is in French...) on singular projection operators are related.
Firstly a comment on each of your "justifications":


*

*
[...] we want to match the eigenvalues of $H$ and $H_{eff}$, and both the left and the right hand sides of the above equation have singularities at the eigenvalues of $H$.

Absolutely not. Instead the singularities of the left hand side (LHS) is a subset of the singularities of the full resolvent $\frac{1}{E-H}$. After all you are projecting onto the subspace.


*
Taylor expanding this equation gives
  $$
\sum_i \frac{H_{eff}^i}{E^i}=\sum_i\frac{PH^iP}{E^i}
$$
  which, by matching powers of $E^i$, gives $H_{eff}^i=PH^iP$, so all powers of the Hamiltonians agree on the subspace.

See my comment to the question. I think this is just wrong, although I am not 100% sure what exactly this expansion is supposed to be.

To the actual question

So, why do we think an $H_{eff}$ will exist?

Because we can find it!
Feshbach derives it in Section II of his 1968 paper.
Projecting the eigenvalue equation in full space $H|\Psi\rangle = E|\Psi\rangle$ onto the two subspaces with projection operator $P$ and $Q$ (where $P+Q$=1, $Q^2=Q$, $P^2=P$, $PQ=QP=0$) we get two coupled equations
$$\left[PH(P+Q)-EP\right]|\Psi\rangle = 0$$
$$\left[QH(P+Q)-EQ\right]|\Psi\rangle = 0$$
and hence
$$\left[E-H_{PP}\right]P|\Psi\rangle = H_{PQ}Q|\Psi\rangle$$
$$\left[E-H_{QQ}\right]Q|\Psi\rangle = H_{QP}P|\Psi\rangle.$$
We can write down a formal solution of the latter
$$Q|\Psi\rangle = \frac{1}{E-H_{QQ}}H_{QP}P|\Psi\rangle.$$
and substitute into the former to get
$$E P|\Psi\rangle = \left[H_{PP} + H_{PQ}\frac{1}{E-H_{QQ}}H_{QP}\right]P|\Psi\rangle.$$
This is a Schrödinger Equation in $P$-space where the subspace state is $P|\Psi\rangle$ and the effective Hamiltonian is
$$H_{eff} = H_{PP} + H_{PQ}\frac{1}{E-H_{QQ}}H_{QP}.$$
That's it! Now you can go ahead and define your resolvents on the subspace and check that it is equal to the projected full space resolvent etc. For projection onto a continuous subspace $Q$ there will be issues with invertibility of $E-H_{QQ}$, but that's something else.
Two good resource I can recommend are Kukulin's textbook and Cohen-Tannoudji's book on Atom-Photon Interactions. The latter has an explicit derivation of why the resolvent in the subspace is the projected resolvent in full space.
Hope that helps, I might have missed the point of this question completely, but I like the topic.
