As you noticed the Laplacian is ill-defined at the origin: such equations are usually solved by lifting them to distribution theory and using Green's functions.
Let
$$
-\nabla^2 \phi(\mathbf{x}) = -4\pi \rho(\mathbf{x})\tag{1}
$$
one can show$^1$ that the solution $\phi(\mathbf{x})$ can always be written as$^2$
$$
\phi(\mathbf{x}) = \frac{1}{4\pi} \int_V d^3 x' \rho(\mathbf{x})G(\mathbf{x}, \mathbf{x}')
+ \frac{1}{4\pi}\int_{\partial V}d\sigma\Big[G(\mathbf{x}, \mathbf{x}')\frac{\partial \phi}{\partial n} -
\phi(\mathbf{x}') \frac{\partial}{\partial n}G(\mathbf{x}, \mathbf{x}')\Big] \tag{2}
$$
where the Green's function $G(\mathbf{x}, \mathbf{x}')$ solves the associated Green's equation
$$
-\nabla^2 G(\mathbf{x}, \mathbf{x}') = -4\pi \delta(\mathbf{x}-\mathbf{x}')\tag{3}
$$
Using appropriate boundary conditions (in the simple cases one can demand the functions to vanish at the boundaries and the first order derivative to be proportional to the surface element$^3$) equation $(2)$ can be solved by plugging the solution of $(3)$, which you already have recognised to be the potential of a single point charge.
Details on all of this can be found in the standard textbook of J. D. Jackson on Classical Electrodynamics.
$^1$ In order to show why this holds multiply $(1)$ by $G(\mathbf{x}, \mathbf{x}')$ and $(3)$ by $\phi(\mathbf{x})$ and integrate against $\delta(\mathbf{x}-\mathbf{x}')$.
$^2$ There might be some $2\pi$ being forgotten left and right, somewhere.
$^3$ I might be wrong about this.