Consider a delta function potential $V(x)=-\alpha \delta(x)$. There is a bound state
$$\psi(x) = \frac{\sqrt{m\alpha}}{\hbar} \exp\left(-\frac{m\alpha|x|}{\hbar^2}\right),$$ with energy eigenvalue $$E=-\frac{m\alpha^2}{2\hbar^2}.$$
However, if we plug the wavefunction into $$E=\langle\psi|H|\psi\rangle,$$ then we get $$E=\left\langle\psi\middle|\frac{p^2}{2m}\middle|\psi\right\rangle-\alpha\langle\psi|\delta(x)|\psi\rangle\\ = -\frac{m\alpha^2}{2\hbar} - \frac{m\alpha}{\hbar^2}\int \exp\left(-\frac{2m\alpha|x|}{\hbar^2}\right)\delta(x)\, dx\\ =-\frac{3m\alpha^2}{2\hbar^2},$$
which is not equal to the eigenvalue. So how do we get around not including the contribution from $\delta(x)$ in $\langle\psi|H|\psi\rangle$?