The potentials and field integrals require that the integrand be integrable, not continuous. Any continuous function is (Riemann) integrable, but an integrable function does not need to be continuous. In fact, for integrals in one dimension, a function is Riemann integrable if and only if it is continuous almost everywhere [that is, everywhere except on a set of (Lebesgue) measure zero]. The condition in three dimensions is more complicated, but for functions that only have discontinuities along the boundaries of simple solids, there is no problem with the integrability of the function.
If you want to calculate the potential and field at a point where the charge density $\rho$ is discontinuous, there is no problem if the discontinuity is a simple one (meaning there is a change from one smooth function to another smooth function across a similarly smooth surface). As an example of this, consider the field inside and outside a uniformly charged solid sphere. The electric field is well defined and continuous at the boundary, as can be verified by Gauss's Law, and the potential is continuous and differentiable at the boundary, meaning that we can indeed calculate the electric field as the gradient of the potential.