E is in the eigenvalue equation as a matter of definition.
Recall that in electrostatics, only differences in potential energy are meaningful. So this concept should not be too unfamiliar to you. It's just that in quantum mechanics we like to use the energy eigenbasis to express the quantum state of the system.
Is it not true that $E$ doesn't factor into any physically meaningful relation, and only $ΔE$ does?
This is true in general, but in the case that $|\psi\rangle$ is an energy eigenstate (NOT a superposition of energy eigenstates) then the expectation value of energy will only depend on the $E$ corresponding to that $|\psi\rangle$. It's the cross terms, or "interference" terms, that contribute the $\Delta E$.
As a specific example, consider the emission/absorption of light quanta: an atom in an initial energy eigenstate will lose energy by emitting a photon and end in an energy eigenstate of lower energy. Thus the photon (the thing we measure) will have an energy that is the difference of the two eigenstate energies.
Where is $E=0$ in this definition of E?
Typically, $E$ is nonzero due to the potential defined by the Hamiltonian (i.e. harmonic oscillator, hydrogen atom, infinte square well...) so this is all just a matter of definition. As Count Iblis said in the comment, if you just redefine your Hamilitonian as
$$ H' = H + uI $$
where $I$ is the identity matrix and $u$ is some nonzero constant, then you'll find that the energies $E'$ corresponding to $H'$ are necessarily nonzero.