You get the solution of $$ \frac{p^2}{2m}\psi_E(p) = E\psi_E(p) $$ as follows
$$ \left(\frac{p^2}{2m}-E \right)\psi_E(p) = 0 $$
For this equation to hold it must be either $\frac{p^2}{2m}-E = 0$ or $\psi_E(p) = 0$. That means for every $p$ except for $p=\sqrt{2mE}$ it must be $\psi_E(p) = 0$. Only for $p=\sqrt{2mE}$ it is allowed that $\psi_E(p)$ is non-zero.
This is exactly the behavior of the [Dirac delta function][1]. So you can write (with an arbitrary constant $A$)
$$ \psi_E(p) = A \delta(p-\sqrt{2mE}) $$ [1]: https://en.wikipedia.org/wiki/Dirac_delta_function