I'm seriously considering establishing a prize for unified field theories in-line with my history of prize awards, such as the fusion legislation I drafted with a cofoudner of the AEC's program, the Bowery Award for Amateur Rocketry (that led the Ansari X-Prize) and The Hutter Prize for Lossless Compression of Human Knowledge.

The rules would be identical to The Hutter Prize rules with one exception:

Replace the enwik9 Wikipedia corpus with physics data.

The value of this prize would be the rigorous and objective model selection criteria:

Solomonoff Induction

Hutter wrote a paper titled "A Complete Theory of Everything (will be subjective)" in which he explains the applicability of Solomonoff Induction to physics, including what it takes to select among any set of UFT's, including those with large numbers of free parameters, such as string "theory". I scare-quote "theory" in the case of "string theory" (and would for any purported UFT with free parameters) because the, somewhat sloppy, vernacular phrase "physical theory" has been taken to mean what the more accurate phrase "physical model" means. In other words, until a physical interpretation is coupled to a theory it remains in the domain of metaphysics if not pure mathematics. Any theory with free parameters remains indeterminate, hence "not even wrong" as there is no interpretation.

A reductio ad absurdum of the view that model selection can take place in the presence of free parameters:

Any "parameter" of unspecified precision, can, via arithmetic coding, provide an unspecified amount of information. A theory with such a free parameter can populate it with an infinite precision floating point number and then, via arithmetic coding, expand it to cover any observations whatsoever.

Expanding on how this would work:

Each school of theoretic thought would create an executable archive of the physics corpus. That executable archive program would have to run within the resource limitations appropriate to the UFT competition. For instance, purpose served by The Hutter Prize is about a week of 1 CPU's time using less than 10GB of RAM is permitted. But other language modeling "competitions" invest millions of dollars of compute resources. Whatever the compute resource choice, the resulting file must match, bit for bit, the original physics corpus.

The shortest program at any given time would be awarded money from the prize purse according to how much it reduced the size from the prior winner's benchmark.

The physics corpus should be adequate to the prize's purpose. For example, The Hutter Prize's corpus, enwik9 (1 billion byte snapshot of Wikipedia) is adequate to its purpose of modeling natural language knowledge.

A critical but inadequate corpus would be The NIST's Fundamental Physical Constants "Value" column encoded as variable precision floating point. An obvious addition to the fundamental constants would be the Ionization Energy column from the NIST Atomic Spectra Database Ionization Energies Data.

I looked around and failed to find any pre-existing such corpus, or list of corpora for testing theories against each other in a rigorously fair manner.

What set of physical data would be adequate as a model selection tool?

  • 1
    $\begingroup$ It seems unlikely that this would make sense. For example, the rules of QED are simple and unambiguous, and in principle can be used to crank out all the atomic spectra. But actually performing this calculation is extremely difficult and requires a vast array of principled approximations. Therefore, if you were judging solely by the size of the program required, a dumb curve fit would beat QED easily, with much less CPU time too. $\endgroup$
    – knzhou
    Aug 17, 2021 at 19:51
  • 1
    $\begingroup$ Furthermore, if you fixed this problem by not penalizing for the complexity of calculating the consequences of a physical theory, there would be a single candidate that completely crushes all others: the Standard Model. But we already knew that. $\endgroup$
    – knzhou
    Aug 17, 2021 at 19:54
  • $\begingroup$ I mistakenly used the Hutter Prize resource limits which are, in retrospect, obviously inadequate. Such limits must be chosen appropriate to the prize. The "curve fitting" aspect of your concern becomes less of a concern the larger the corpus of measurements and the larger the computational resources permitted. The large language models (GPT etc) require millions of dollars of compute. Think in those terms. $\endgroup$ Sep 23, 2021 at 16:03

2 Answers 2


Comparing theories in this way is not likely to be relevant.

Consider the actual candidates for theory of everything. We have string theory, which is uniquely compelling as a step beyond field theory, but whose empirical application requires searching an ill-understood googol-sized landscape of possibilities for worlds that appear to resemble our own, and then calculating the detailed properties of physics in each such world. Machine learning is being used to conduct the search for qualitatively promising worlds, but the second step, of detailed calculation, requires much more progress in fields like algebraic geometry.

Then we have all the theories that aren't string theory: like, loop quantum gravity coupled to a grand unified field theory, or celebrity theories of everything like those due to Lisi or Weinstein, or dozens of similar but unheralded proposals that have been posted to arxiv. None of these theories can calculate everything either - indeed, most of them can't calculate much at all. They're all "works in progress".

On the other hand, you have extensions of the standard model that include new fields and make slightly different predictions. Like the standard model, these are field theories with numerous free parameters. Conceivably you could rank such theories by the number of bits required to specify them, and the number of bits of the standard model parameters that they would explain; but such theories generally make entirely new predictions too (e.g. proton decay), and how they fare on that front is more important for their ultimate credibility.

  • $\begingroup$ Any "theory" that, due to unspecified parameters, is incapable of calculating predicted observations can only be considered to represent a class (distribution) of theories. The point of the prize I posit is not to select among classes of theories but to select from among theories that have interpretations -- models that best fit the evidence in terms of the algorithmic information criterion for model selection. This doesn't stop "works in progress". Indeed it could finance them if sufficiently underwritten. $\endgroup$ Dec 24, 2022 at 13:50
  • $\begingroup$ Well, we have a model - the standard model - and it fits all the particle data so far; and all predictions of anything in addition to standard model particles, have failed. Furthermore, known frameworks capable of determining what are free parameters within the standard model framework, do predict new particles. So for now, the standard model, with its free parameters set to their observed values, is undoubtedly the most economical theory that accounts for the data... $\endgroup$ Dec 25, 2022 at 0:29
  • $\begingroup$ The main complicating factors come from astrophysics - the abundance of matter over antimatter, and dark matter. One may account for these by adding new particles, as seen in various minimal extensions of the standard model, but it's also possible that "dark matter" is more accurately thought of as a modification to the gravitational field... $\endgroup$ Dec 25, 2022 at 0:32
  • $\begingroup$ Cosmology might actually be the domain in which your approach has the best chance of being relevant, though because of the variance of astrophysical objects, one is instead aiming for the best and most economical fit to the data, rather than seeking the shortest program which reproduces it all exactly, bit for bit. $\endgroup$ Dec 25, 2022 at 0:34

I think there are some features of the problem of predicting various measured quantities from principles, from a computer science perspective, that make it unlike predicting text from a natural language model. Here are a few (there is some overlap between them):

  1. We never know the ground truth in physics. For natural language processing, there is a clear goal: the text of wikipedia is a snapshot of how people write English today, and can serve as a ground truth for a NLP model. In physics, we don't know many crucial experimental facts that would be needed to distinguish between candidate models at the Planck scale. Is there supersymmetry broken at some scale? If so, which scale? What is the scattering cross section of dark matter with neutrinos? What is the fourth digit of the quartic self-coupling constant of the Higgs Boson? We don't know these quantities, but surely they will be needed to distinguish candidate models. Perhaps there are errors in the current knowledge of fundamental constants, so that a theory that matches the list too well will actually turn out not to be a good one in the end.

  2. Computational complexity is not a proxy for truth. While we want the simplest model that will fit the data, there is no way to put an absolute bound on the computational resources needed for an accurate model. A Newtonian gravity simulation would take far fewer resources than a full GR simulation, but to accurately model gravitational waveforms we surely need the latter. Such a simulation pushes the boundaries of what is capable with supercomputers today, but this should not be taken as a point against GR, compared to a phenomenological fit that could reproduce observed gravitational-wave data. In fact, simulating a quantum mechanical system with $N$ qubits takes a very large number (exponential in $N$) of classical resources.

  3. Not all insights are data driven. MOND does an excellent job of fitting galaxy rotation curves. However, by itself, we know it is not a complete theory because it is not relativistic. There are proposals to make MOND relativistic, and I don't want to get into an argument about whether MOND is correct or not, but the point is that there are criteria beyond comparing to data that guide us to a correct description. I suspect a potpourri of different phenomenological models that fit different data in the corpus could do a very good job fitting a set of physics measurements, but would certainly not be acceptable as a fundamental theory of physics.

  4. The value of physics is to be predictive outside of the scope of the original data. Data-driven approaches like machine learning work well when the training set covers a representative range of cases that cover likely data that will be seen in the future. Said differently, statistical models are good at interpolation but poor at extrapolation. On the other hand, the value of fundamental physics is precisely to make predictions about situations that have never been seen before. Nothing in our everyday experience could make us think it would be reasonable to predict the state of the Universe 3 minutes after the Big Bang, for example. But taking our theories of GR and nuclear physics and extrapolating them in a conservative way, we are able to model this process and make predictions that are later confirmed by experiment. Testing theories by how they extrapolate into new areas is a crucial part of the development of physics, but is something that would not be possible in this kind of approach.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.