You get the solution of
$$ \frac{p^2}{2m}\psi_E(p) = E\psi_E(p) $$
as follows

$$ \left(\frac{p^2}{2m}-E \right)\psi_E(p) = 0 $$

For this equation to hold it must be either $\frac{p^2}{2m}-E = 0$ or $\psi_E(p) = 0$.
That means for every $p$ except for $p=\sqrt{2mE}$ it must be $\psi_E(p) = 0$.
Only for $p=\sqrt{2mE}$ it is allowed that $\psi_E(p)$ is non-zero.

This is exactly the behavior of the [Dirac delta function][1].
So you can write (with an arbitrary constant $A$)

$$ \psi_E(p) = A \delta(p-\sqrt{2mE}) $$

 [1]: https://en.wikipedia.org/wiki/Dirac_delta_function