Why should dark matter and dark energy inherently require theoretical modification?

Question

What we know more or less directly about both dark matter and dark energy is their spacetime curvature, and attempts to detect corresponding particles have thus far come up short as far as I've read. So why should we assume these two phenomena are anything more than spacetime curvature?

Variations of this question have been asked and answered on this forum, so let me try to preempt some possible answers so as to hopefully push the debate a bit further.

The most convincing counter-argument I've seen is as follows: according to GR, spacetime curvature must be caused by matter, and according to QFT, matter is best represented as particles/quantum fields, hence dark matter ought to be another particle.

My response: the first half of that argument is pointless, because GR in its infinite wisdom places no restrictions of its own on what matter is. It just says that, since the Einstein tensor is identically conserved locally in any Riemannian/Lorentzian manifold, and matter (mass-energy-momentum) is exactly conserved in the Newtonian limit, then matter should be that which flows along with the Einstein tensor. But it doesn't say that there should always be a physical entity beyond the Einstein tensor itself, i.e. spacetime curvature. As far as GR is concerned, a curved Lorentzian spacetime is a matter-filled universe in its own right.

So then it is up to QFT to argue that the "anomalous" dark spacetime curvature we see is due to one or more particles. Now this is where I have a lack of expertise, but my thinking is that quantum theory was developed specifically to explain small-scale phenomena that we already knew included particles. But, and especially because we know quantum theory is not the final answer, why should we suppose it extends to completely new phenomena that can only be perceived on cosmic scales? Sure, there are hypotheses that e.g. vacuum fluctuations can explain the CMB pattern or something (again, that's all pretty unfamiliar to me), but my impression is that those are more attempts to fit quantum theory to big bang cosmology rather than something that is unequivocally predicted by quantum theory.

The natural comeback is then: okay, but then why should the spacetime curvature regions known as dark energy and dark matter follow the behavior patterns they do? To wit: dark energy is homogeneous and isotropic, and dark matter formed the cosmic web that proceeded to attract the nascent galaxy clusters into the present-day state of the universe. Dark matter also forms halos around large gravitating clusters, yet can detach from them as in the bullet cluster collision.

My response: if, as argued above, GR in its purest form allows spacetime curvature to exist without some external causal agent, then the Lorentzian manifold corresponding to the distribution of dark energy and dark matter requires no explanation. It, rather than Minkowski space, is the spacetime in which quantum fields live. And after all, why should we suppose that quantum fields would live in Minkowski space? This argument likewise applies to cosmic inflation and any other regions of exotic curvature we may yet find. It would also imply, for example, that, rather than dark and baryonic matter being mutually attracted, dark matter simply exists, and baryonic matter follows it through spacetime due to the principle of geodesic motion.

And here's the final retort I imagine you making, to which of course I have no real counter: okay, of course we can model the universe as a QFT in curved spacetime, but, as we all know, that is not complete. A perfectly accurate theory would ultimately have to unify QFT and GR, not just combine them. And if the unified theory must be quantum in nature, then any regions of curved spacetime must become describable as quantum fields.

But here's my point, I think. If we do accept that the best current theoretical model of the universe is QFT in the darkly curved spacetime, doesn't that force us to think twice about what a unification theory ought to look like? If, as far as our observations can tell, spacetime curvature with no accompanying quantum fields is just as fundamental as the quantum fields themselves, then doesn't that tend to shift the balance just a little further towards a paradigm in which the spacetime continuum itself plays an indispensable role?

Of course, a proper unified theory would almost certainly construe baryonic and dark matter as mutually attracting, which is what the dark matter particle candidates are meant to do. And this will have to come from whatever mechanism couples particles to spacetime curvature. And naturally it will have the benefit of more intimately interweaving the stories of particles and spacetime, such that each is given its full account only in terms of the other. But my frustration is that I don't believe we should a priori treat it as necessary that these "dark" phenomena have some underlying causal justification, because to me that feels like a slap in the face to the great gift to physics that is general relativity -- not only the merging of space with time, but the letting go of Euclid's fifth postulate that had constrained our scientific thinking for so long, such that the continuum became free to twist and turn in an infinite variety of ways.

Perhaps I am only venting my philosophical frustration after all, but I would genuinely appreciate a response.

Answer 1

In the classical limit, the laws of physics should be causal and deterministic, i.e., initial and boundary conditions should uniquely determine the behavior over spacetime, up to diffeomorphisms (and gauge transformations). This is generally achieved by writing a classical field theory in terms of a principle of stationary action.

In the Einstein field equation $G^{\mu\nu} = 8\pi T^{\mu\nu}$, the left-hand side comes from variation of the gravitational action and the right-hand side comes from variation of the matter action -- each variation being with respect to the metric $g_{\mu\nu}$. For a deterministic solution, we must have some specific form of the stress-energy tensor $T^{\mu\nu}$.

If all we specify about $T^{\mu\nu}$ is that it obeys the local conservation law $\nabla_\mu T^{\mu\nu} = 0$ and is otherwise unrestricted, then this doesn't tell us anything about the geometry of spacetime, because $\nabla_\mu G^{\mu\nu} = 0$ is a mathematical identity. To have a deterministic theory, we should explicitly derive $T^{\mu\nu}$ from a matter action, and should also include the matter equations of motion from varying the matter action with respect to the underlying matter fields.

A rough argument that such a theory achieves determinism is as follows: Each matter field has a corresponding equation of motion from varying the matter action with respect to that field. Meanwhile, there are 10 independent components of the metric $g_{\mu\nu}$, whereas $G^{\mu\nu} = 8\pi T^{\mu\nu}$ provides only 6 independent equations for these components (because applying $\nabla_\mu$ to both sides shows that 4 of the nominal 10 components of the Einstein field equation are identities). The overall result is that we have 4 fewer equations than unknowns, which is exactly right to allow for an arbitrary diffeomorphism with 4 components that does not affect the physically observable behavior.

Thus, I disagree with the statement that

GR in its purest form allows spacetime curvature to exist without some external causal agent

since a very important property of GR is its determinism in the presence of a well-defined matter action (including the absence of matter, giving $G^{\mu\nu} = 0$, which is arguably actually "GR in its purest form").

Note 1: Technically, some things can be said about the geometry of spacetime under inequality constraints on $T^{\mu\nu}$ known as "energy conditions", which are fairly general. The resulting conclusions, though, are about overall structural features (such as singularities) and do not by themselves provide a deterministic solution.

Note 2: A degenerate case where we can write a "matter action" without any matter fields is the cosmological constant $\Lambda$: The matter action is proportional to the spacetime volume, and the stress-energy tensor is $T^{\mu\nu} = \Lambda g^{\mu\nu}$. This automatically satisfies $\nabla_\mu T^{\mu\nu} = 0$ without requiring any matter equations of motion. It can be also be considered an addition to the gravitational (as opposed to matter) action.

Answer 2

I think the answer is simple, and the problem with your post is that it has too many words and not enough equations. Sure, GR allows for vacuum solutions - that is, spacetime curvature in the absence of matter. But the particular solutions that we observe in our universe (through many observations such as galaxy rotation curves, galaxy cluster speeds, the CMB, large scale structure, and so on) are not vacuum solutions.

This is not something that we postulate: you can take the metrics corresponding to all these observations, plug them into the Einstein equation, and find that the right hand side is not zero. If we accept Einstein's theory, we are forced to include dark matter - actual matter, not in a philosophical "curvature can also be considered matter" sense.

In other words, you're suggesting that in the past eighty years or so, no one bothered to actually check mathematically if the gravitational fields they were observing required the presence of matter. Does that sound likely to you?

Answer 3

Two things have always wondered me concerning dark matter: First, that our galaxy rotating faster than it should rather points to us not understanding perfectly how gravity works on a large scale yet (compare this with dark energy) instead of us doing so and there being an unknown type of matter. Second, if this type of matter really does exists and its only interaction with other matter is with gravity, then it is not possible to detect it otherwise. Also describing it with QFT would not make any sense at all. There are multiple theories that alter the field equations in a way to explain the phenomena of dark matter and dark energy without an additional source.

Concerning GUT, I think that the approach of string theory to describe multiple particles as different vibrations of the same string points in the correct direction. Physics has often shown that what we thought was different turned out to be the same, for example energy/matter, electromagnetism and spacetime. Another example is the idea of a dark fluid unifying dark matter and dark energy, that states they are neither seperate phenomena nor have seperate origins but are instead linked together as something called a dark fluid: On a small scale it produces more attraction similar to dark matter and on a larger scale it produced more repulsion similar to dark energy (See here).