In Carroll's Appendix B, he says
You will often hear it proclaimed that GR is a "diffeomorphism invariant" theory. What this means is that, if the universe is represented by a manifold $M$ with metric $g_{\mu \nu}$ and matter fields $\psi$, and $\phi : M \to M$ is a diffeomorphism, then the sets $(M, g_{\mu \nu}, \psi)$ and $(M, \phi^* g_{\mu \nu}, \phi^* \psi)$ represent the same physical situation. ... This state of affairs forces us to be very careful; it is possible that two purportedly distinct configurations (of matter and metric) in GR are actually "the same," related by a diffeomorphism.
I completely agree that two pseudo-Riemannian manifolds $R' = (M', g')$ and $R = (M, g)$, where $M', M$ are smooth manifolds and $g', g$ are metric tensors, are physically equivalent iff there exists a diffeomorphism $\phi:M' \to M$ such that $g' = \phi^* g$ (and, if there are matter fields, $\psi' = \phi^* \psi$, but for simplicity I'll focus on the vacuum case). However, I think his use of the phrase "related by a diffeomorphism" to describe this relation is a bit misleading. In the standard mathematical usage, a "diffeomorphism" is an isomorphism between smooth manifolds (the $M$ and $M'$) and doesn't "touch" the metrics at all. One can consider two Riemannian manifolds $(M, g)$ and $(M, g')$ which are diffeomorphic but have completely independent metric structures. For example, consider the flat unit disk in the $x$-$y$ plane and the upper unit hemisphere, both embedded into $\mathbb{R}^3$ and inheriting the usual Euclidean 3-D metric via the standard pullback mechanism. These Riemannian manifolds are diffeomorphic in the standard mathematical sense, but not "related by a diffeomorphism" in the sense that Carroll describes in the quotation above, because the metrics are not related by the relevant diffeomorphism pullback.
The relation between pseudo-Riemannian manifolds that Carroll describes, in which the metrics "agree" via the relevant diffeomorphism pullback, appears to be what mathematicians call an isometry, which is a very special case of a diffeomorphism. A (mathematicians') diffeomorphism is the natural notion of isomorphism between smooth manifolds, but a (mathematicians') isometry is the natural notion of isomorphism between (pseudo-)Riemannian manifolds - not only the smooth structure but also the metric structure gets "carried over" appropriately.
Question 1: Am I correct that using standard mathematical terminology, the transformation that Carroll is describing an "isometry" rather than a general "diffeomorphism"?
Putting aside Carroll's particular choice of phrasing, I believe that isometry between pseudo-Riemannian manifolds (in the standard mathematical usage linked to above, which is not the same as the usual physicists' usage) is actually the correct notion of physical equivalence in GR, rather than general diffomorphism. As discussed here, general diffeomorphisms do not map geodesics (which are physical and coordinate-independent) to geodesics - only isometries do. Moreover, in my disk and hemisphere example above, the former manifold is flat and the latter is curved, so on the latter surface initially parallel geodesics meet, triangle corners add up to more than $180^\circ$, etc. These non-isometric manifolds clearly correspond to distinct physical states, even though they are diffeomorphic.
Question 2: Am I correct that two Riemannian manifolds correspond to the same physical state iff they are isometric, not merely diffeomorphic (again, under the standard mathematical definitions of "diffeomorphism" and "isometry", not under Carroll's definitions)?
Those are my physics questions. If the answers to both questions #1 and #2 are "yes", then I have a closely related usage question. It seems to me that Carroll's usage of the word "diffeomorphism" is not a personal quirk or sloppy language, but is standard in the physics community. Many times, I've heard physicists say that diffeomorphic Riemannian manifolds are physically equivalent, or that GR is "diffeomophism-invariant".
Question 3(a): When physicists talk about a "diffeomorphism" in the context of general relativity, are they usually using the word in the standard mathematical sense, or in Carroll's sense, which mathematicians would instead call an "isometry"?
If the answer is "in Carroll's sense", then that means that the mathematics and physics (or at least GR) communities use the word "diffeomorphism" in inequivalent ways. This wouldn't surprise me, except in that if so, I've never heard anyone mention that fact.
Question 3(b): Physicists often say that the theory of general relativity is "diffeomorphism-invariant". Am I correct that this is true under the physicists' usage, but under the mathematicians' usage, GR is not diffeomorphism-invariant but only isometry-invariant?