In Feynman lectures, Volume 2 chapter 8 (https://www.feynmanlectures.caltech.edu/II_08.html#Ch1-audio) at the very end Feynman remarks
We must conclude that the idea of locating the energy in the field is inconsistent with the assumption of the existence of point charges. One way out of the difficulty would be to say that elementary charges, such as an electron, are not points but are really small distributions of charge
His reasoning for this was $$U=(\epsilon_0/2)\int E^2 dv=\int_0^\infty\frac{q^2}{8\pi r^2\epsilon_0}dr=\infty$$
One may think to calculate the self energy of a point particle by the product of the charge of this particle times the potential caused by this charge itself at its own position that is $r=0$, thus we get $$U=Kq²/0=2\infty$$ Here $K =1/4\pi\epsilon_0$ and the factor of 2 signifies that this is double of the Feynman's calculation.
However this approach is fundamentally flawed which one can see as soon as they realize that a particle cannot apply a force on itself.
But instead of this we can think of a point particle to be made of many infinitesimal charges, $dq$, then we can bring back our potential energy model since here any charge $dq$ would be experiencing force from all the other particles and not itself (though this "self force" is included in the integral below but is of no consequence since it is of second order), thus we can write $$U=(1/2)\int\phi dq = q\phi/2 = \infty$$
Where $\phi$ is the potential of the charge ($\phi=Kq/r$) and is pulled out of the integral since it is constant over an infinitesimal volume we are considering. This is the same result as Feynman's without using the model in which energy is stored in fields (well electric field, that is one can always argue that it's stored in potential field, but then the argument becomes redundant).
So what is Feynman trying to imply/justify here since using both models we get inifinity in the end, how can infinity of electric field model be enough to discredit it?
Note: Though trivial and obvious it is important to realize that $dq≠q $, where $q$ is charge of point particle and that energy is always $+\infty$ no matter the sign of charge.