The abstract space of metrics in GR I know this is a general (har har) question, but has any work been done understanding the mathematical space the allowed metrics in GR form? (I guess it'd be called a tensor space???)
 A: Sure, this kind of thing is well studied. The "space of metrics" of a manifold is usually called its moduli space. More precisely, if we have a manifold $M$, Let $\text{Met}(M)$ be the set of all possible metrics on $M$ and define an equivalence relation $ \thicksim $ on it, where metrics $g, g'$, are considered the same if there is an automorphism (diffeomorphism, biholomorphism etc depending on what structure on $M$ we are interested in preserving) of $M$ that takes $g$ to $g'$. The moduli space of $M$ is then defined to be $\text{Met}(X)/\thicksim $, and in most cases it can be given at least the structure of a "manifold with corners". I don't think this full-fledged space is that important in GR, since in GR we are for the most part only interested in metrics that obey Einstein's equations. However, its of fundamental importance in string theory, where one integrates over moduli space of Riemann surfaces. 
A: Another sense of spaces in which took for metrics is in the initial value formulation of General Relativity.  Just as with a wave equation you can specify the wave and it's time variation at an initial surface and then find a solution through the domain of dependence , so can you specify a surface with a form and some structures to say how they are "moving" and then grow that into a full 4D solution to Einstein's Equation that appropriately contains that surface and it's motion as a slice.
And the interesting thing is that unlike the relatively simple wave equation in a pre-existing flat background space, there are much more interesting restrictions on the space of initial conditions that determine which can be consistently turned into 4D solutions.
