"It is the gist of general relativity that it admits, on an equal footing as it were, every possible coordinatization." The title is a quote from Hermann Weyl in a 1955 article:

Weyl, Hermann. 
  "Why is the world four-dimensional?" In
  Levels of infinity: Selected writings on mathematics and philosophy. Courier Corporation, 2013. p.204.

Could anyone please explain what he means by that sentence? I realize it may be difficult without
the surrounding context, but perhaps it is already clear to those immersed in the
topic. 
 A: Since the OP is a mathematician, I'll write this in a style that I would think would work for a mathematician with no specific background in physics.
Newton claimed:

Absolute, true and mathematical time, of itself, and from its own nature flows equably without regard to anything external, and by another name is called duration: relative, apparent and common time, is some sensible and external (whether accurate or unequable) measure of duration by the means of motion, which is commonly used instead of true time ...

This is sort of a similar outlook to the one created by Descartes a generation earlier. We imagine space and time as imbued with preferred coordinate systems, such as Cartesian coordinates for the Euclidean plane. Although there may be some freedom to choose our coordinates, e.g., we can rotate or translate the axes, they are imagined to be rigid. There can be coordinates that are not as good, because lines don't have equations of the expected form.
In coordinates $(t,x)$, Newton's laws make predictions about $d^2x/dt^2$. Therefore we can always make a change of coordinates of the form
$$ x' = x+vt+x_0$$
$$ t' = t+t_0,$$
and because the second derivative stays the same under this transformation, the laws still work. However, there are other coordinates that we're not allowed to use, e.g., if $(t,x)$ are coordinates in which Newton's laws hold, then we're not allowed to use coordinates such as
$$ x' = x^3$$
$$ t' = t.$$
These attitudes about the absolute nature of time and space became very ingrained, but general relativity pretty much demolished them. GR takes place on a manifold, and you typically can't even coordinatize an entire manifold using one set of coordinates. Therefore we end up expressing GR (in the standard formulation) entirely in terms of tensors, in such a way that any equation remains true under any smooth change of coordinates (diffeomorphism).
The attitude is that coordinates are just names for points in spacetime, and the points don't care what we name them.
Many physical theories other than GR can also be expressed in this way, so I don't really agree with Weyl'ls characterization in detail. But it is true that some physical theories resist being cast in this way. For example, in quantum mechanics time is a parameter rather than an observable, so it can't be treated on the same footing as space, and we don't have the freedom to subject time to an arbitrary smooth change of coordinates.
A: It's known as Principle of General Covariance. It was coined by Einstein, but I now understand that Weyl too considered it (in 1955!) a significant statement. To me it has long appeared the most vacuous principle of Physics.
Actually it holds for any physical theory. For the moment think of
classical theory of fields (electromagnetism, for example). There's
nothing against expressing it in the most general spacetime
coordinates, as already noted by @BenCrowell. Yet it took a long time
before theoretical physicists understood this - Einstein and Weyl
weren't alone. It would be an interesting topic for history and
philosophy of science, but this isn't the right place to delve deeper.
The first book explicitly contrasting general covariance I knew of
is Gravitation by Misner, Thorne, Wheeler (Freeman 1973). There you may read (p. 431)
The "no-priori-geometry" demand actually fathered general relativity, but by doing so anonimously, disguised as "general covariance", it also fathered half a centry of confusion.
In the same book Ch. 12 is entirely devoted to Cartan's (1923)
Newtonian spacetime, a curved stratified (not Riemannian) 4D
manifold describing Newtonian gravity with absolute space and time.
Of course in such case there is a natural, intrinsic way of defining a $t$ coordinate and a Euclidean 3D space. Then it's justified to adopt $(x,y,z)$ coordinates (defined up to a Euclidean transformation) and those $(t,x,y,z)$ are the simplest environment where to develop Newtonian mechanics. (Not forgetting that there are several cases where different space coordinates - e.g. polar ones - are more useful.)
A final point about (non-relativistic) QM. What Ben observes as to the special rôle of time is true. But I have the vague idea that if one so desired a different choice of coordinates would also be possible - if less handy. Something like Cartan's treatment of Newtonian gravity. I would add that Euclidean structure of space is understood in QM. $(x,y,z)$ are not only and always operators, their rôle as coordinates isn't canceled, I believe.
It would also be interesting to consider under this point of view
Dirac's theroy. It's relativistic, yet it's a QM in its own right. How are these two requirements reconciled?
