Should the potential of a point charge be taken from the Hubble boundary instead? The expansion of space yields a limit to our observable universe, somewhere roughly 13.8 billion light years away. Past this point, galaxies and any other particles are expanding away from us faster than the speed of light due to the expansion of space over such a vast distance.
There is no hope of those objects ever having any causal effect on anything within this Hubble radius relative to your view here on Earth with current physics. Because there is an observable limit to our universe, isn't it actually correct to calculate the potential of a point charge not as an integral from infinity, but as an integral from the Hubble boundary, since the Hubble boundary is the potential limit that to what could ever have any relevant physical meaning? 
 A: It really doesn't matter.
So for example for the Sun, the error is, what? The Sun's radius, while large, is 700,000 km; the Hubble radius is 14 billion light years. So that's a correction to the gravitational binding energy of the Sun of something like one part in 190,000,000,000,000,000. Even if you scale up to the size of the Milky Way Galaxy the error is at most, say, one part in a million, if that.
For comparison our uncertainty in $G$ is one part in twenty thousand or so, if memory serves me correctly. 
A: Potential energy is defined relative to some more or less arbitrary zero level, since usually only differences in potential matter. Taking zero to be at spatial infinity is usually convenient. Since the limit of the observable universe is changing with time, using that as zero would make potentials inconveniently time dependent.
It is worth noting that the Hubble boundary might not even be the one you are thinking of. There is smaller distance (about 5 Gpc away in co-moving coordinates) that represents the maximum distance anything we do can affect, and a larger distance (about 55 Gpc) that represents the eventually observable universe. See (Davis & Lineweaver 2004) for a nice overview of some of the different horizons. 
Davis, T. M., & Lineweaver, C. H. (2004). Expanding Confusion: common misconceptions of cosmological horizons and the superluminal expansion of the Universe. Publications of the Astronomical Society of Australia, 21(1), 97-109.
https://arxiv.org/abs/astro-ph/0310808
