Gravitational potential has a "floating" value, something like voltage. Any arbitrary point can be chosen and its gravitational potential can be called "zero". The gravitational potential at any other point in the universe is then the difference in gravitational potential between the chosen point and the other point. The same principle applies to electrical potential.
"Potential" is different from "potential energy". Potential is the energy (difference) per unit mass or unit charge depending on whether we're talking about gravity or electromagnetism. Potential energy of a mass is the gravitational potential multiplied by the mass; potential energy of a charge is the electrical potential multiplied by the charge.
To measure gravitational potential at a point X, we can (in principle) place a tiny mass at the "zero" point and let it fall to X. At X we measure the kinetic energy of the tiny mass. All of the kinetic energy came from the change in gravitational potential energy, so we divide the kinetic energy by the tiny mass, and that is the negative of the change in gravitational potential energy.
An alternate way to measure gravitational potential at X is to move the tiny mass slowly from the zero point to point X, and measure the direction and magnitude of force that must be exerted on the mass all along its path. The net work done by the mass in getting from "zero" to X is the integral of the dot product of the force with the unit tangent vector to the path, all along the length of the path. That work is equal to the nega change in gravitational potential energy, so the change in gravitational potential is obtained by dividing by the mass. This can be a complicated measurement. However, if the measurement is done, the result is the same as the first method.