What scientific evidence or reasoning the authors (Lee Smolin et al) of the "Autodidactic Universe" (article) provide to support their hypothesis?

Question

In their article The Autodidactic Universe (Lee Smolin et al), the authors propose that the fundamental laws of nature may have evolved since the Big Bang, akin to a Darwinian process. Do they suggest that these laws continue to evolve over time? What scientific evidence or theoretical arguments do they present to support this hypothesis, and how does it relate to established physical theories of cosmic development?

Answer 1

1. The state of the Universe is governed by a function.

2. Neural networks approximate functions -- it is what they are designed to do. Period.

3. Therefore, the Universe is an approximation of some ideal and unattainable function ... wait, what? Why? Why bother? Why do we need an approximation when the real thing will do? There is an acronym -- KISS -- keep it simple, stupid.

The only reason they use the word autodidactic is so that they can skip the question of who's doing the training -- God? Ancient astronauts?

It is the cherry picking of concepts from computer science, is what it is. Nowhere do they mention the use of the logistic activation function, or activation functions in general. Weird.

In any case, they're years behind in their research. The latest and greatest toy is the Convolutional Neural Network (CNN). For instance, dog vs cat classification fails 50% of the time using a traditional neural network. The failure rate is 10-20% for the corresponding CNN. They only mention CNNs in passing. In other words, their model is out-of-date out-of-the-gate.

Answer 2

Machine learning has borrowed a lot of ideas from physics, as this year's Nobel Prize in Physics testifies. There is traffic in both directions; for example, physics tools like "mean field theory" may be used in an attempt to understand the complex dynamics of neural networks; while various kinds of physical systems (Ising spin systems, gauge fields, ...) have been proposed as prototypes of new neural network architectures.

This paper considers a number of paradigms for fundamental physics, whose models can also be interpreted as learning systems. For physicists, the most familiar paradigm employed in the paper might be, a theory in which there are many different "vacua" or ground states, each with its own effective dynamics. An example of this could be a grand unified theory with a number of Higgs fields whose potential has many local minima. The symmetry can therefore be broken in many ways, and there may be tunneling between these different metastable vacua. The same phenomenon of an energy landscape with many local minima is very familiar from machine learning.

The philosophical question is then, when does dynamics under these conditions count as learning? The paper proposes that there needs to be a "consequencer", defined as a sub-process that "accumulates information from the past that is more influential to the future than is typical for other contents of the system". They list the consequencers for the models they consider on page 33. For example, they say

Renormalization is by definition a consequencer generator provided it is part of a feedback dynamic. If renormalizations are relevant to the ongoing evolution of a system, then that system is driven by an exemplary consequencer. Renormalization in that case partitions the most causally relevant features of the system.

This sounds promising; and in the preceding section, they specify a kind of model (cubic matrix model) which is also meant to be isomorphic to a kind of neural network architecture that they define (cubic learning system). Unfortunately, they never seem to arrive at the point of exhibiting a particular cubic matrix model whose renormalization group evolution clearly contains a consequencer. And the other models of physical dynamics that they consider (e.g. precedence dynamics) do not have anything like the bonafides of matrix models.

Here I will emphasize that I may have missed something crucial. I haven't patiently read the paper through from start to finish, I've been jumping around trying to get the gist. Maybe I missed the part where they specify something that is both plausible as a theory of everything, and clearly a learning system by their own definition. But so far, I don't see it.

However, I think the paper is valuable as an attempt to be logical and explicit, regarding a hypothesis that is "in the air", given the back-and-forth between fundamental physics and machine learning that I mentioned earlier. It is the most thorough and systematic development of the idea that I have seen, in particular in its consideration of what kind of dynamics could reasonably be counted as "learning". It does therefore deserve to be studied by anyone considering the idea of a universe where the laws of physics learn, whether from a speculative or a skeptical standpoint.