I've read that Newton determined the average density of Earth is twice the density of the surface rocks, but I can't find his computation anywhere. I presume he used the differential calculus in some way. This is the correct result.
The density of granite is 2.765 g/cc, so the density of Earth should be about 5,530 kg/m^3.
https://www.nature.com/articles/ngeo954
For a spherically symmetric planet ( concentric shells of density that vary with radius), Gauss's Law says the gravity you feel at $r_0$ depends only on the mass at $r<r_0$. The force on you depends only on the Mass enclosed by the Gaussian surface.
So if you go down a 100m mine shaft, the gravity from the top 100m of the entire planet has no affect on you.
Any mass, $M$, below you acts as if it were concentrated at the center, with force per unit mass:
$$ g = G\frac M {r_0^2} $$
So for a spherical Earth, the gravity at the surface is the same if all the mass is at the center.
The acceleration due to gravity at the surface is:
$$ g(r) = G\frac M{r^2}$$
If we reduce $r$, there is less mass $M$ pulling on us, but it is closer, so it pulls harder. 100 meters down g is essentially g at the surface.
At this point we use the quotient rule to find $\frac {dg}{dr}$, and set it equal to zero, to find a critical value.
$$\frac{dg}{dr}=\frac{G}{r^2}\left(\frac{dM}{dr}-\frac{2M} r \right)=0$$
or
$$\frac{dM}{dr} = \frac{2M} r $$
The LHS is the surface area times the surface rock density:
$$\frac{dM}{dr} = 4\pi r^2\rho(r)$$
For the RHS, the mass is the volume times the average density of Earth:
$$ M =\frac {4\pi} 3 r^3 \bar{\rho}$$
Therefore
$$4\pi r^2\rho(r) = \frac {4\pi} 3 r^3 \bar{\rho}\frac 2 r$$
Therefore
$$\rho(r) =\frac 2 3 \bar{\rho}$$
This isn't
$$2 \rho(r) = \bar{\rho}$$
Taking the result at face value, the gravitational constant is
$G=\frac{g}{2765 2πR_{earth}}$
So
$G=8.85408 \text { X } 10^{-11} \frac{m^3}{kgs^2}$
I noticed that this is almost the permittivity of free space $\epsilon_0 = 8.854187817 \text{ X } 10^{-12} \frac{C^2}{Nm^2}$.
If you define the permeability of free space as $\mu_0 = 4\pi \text{ X }10^{-8} \frac{N}{A^2}$ instead of $4\pi \text{ X }10^{-7}$, then you get the result that
$$G=\epsilon_0$$
Forgive the units.
Could someone be playing games with the physical constants for some reason unknown to me? Gauss's law is pretty solid.