Number of bits needed to express physical laws?

What is the minimum number of bits that would be needed to express a given physical law, like the law of universal gravitation? How many bits are needed to express each of the four fundamental forces? Is there a pattern here?

-
It would probably depend heavily on whether you assume such things as human language, common experience, etc. known or not. – Alexey Bobrick Jun 20 '12 at 21:11
To extend Alexey's comment a bit: the question as it is has no real answer. If I take a compression algorithm and encode every possible symbol as 01, 001, 0001, 00001, ... a single bit 1 can stand for a physical law but the information is implicit in my dictionary which depends on the knowledge I assume as given. – Alexander Jun 20 '12 at 23:15
Alex, thanks for that. By bit I mean an answer to a yes/no question. Another way to phrase my question is to ask: What is the minimum number of yes/no questions that would be needed to compute the law of gravitation? For example: Question 1: Is the mass of object A m? Yes/No Question 2: Is the mass of object B m? Yes/No Question 3: Is the distance between A and B d? Yes/No If objects A & B both have mass m and the distance between them is d then one can execute the law of gravitation (m1*m2)/d^2 using only 3 bits of information. Guesses for m & d would have to be right, which is improbable. – Professorpedia Jun 21 '12 at 3:31
@Professorpedia: This is the size of the RAM used in the simulation, and in general, to accuracy $\epsilon$, you need $\log(\epsilon)$ yes/no questions to determine the bits. – Ron Maimon Jun 21 '12 at 14:51

2 Answers

One productive way of thinking of the complexity of physical laws is in terms of the Kolmogorov complexity of the algorithm that simulates a given physical situation. This is defined as the length of the shortest code which does the simulation.

If you are given a law of nature, like Newton's law of universal gravitation, you can write a simulation of N-interacting particles. If you are only interested in an in-principle answer, you are looking for the best algorithm to simulate Newton's laws on a computer.

The problem of computing the Kolmogorov complexity precisely is in general the worst of all uncomputable problems. You can usually shrink a code written from scratch by a lot by cleverly rewriting the subroutines to make a specialized language for the description. You would never know if you have the optimal coding, since maybe you could compress things more by adding a special interpretation layer.

The rigorous version of this annoyance is the theorem that no axiomatic system can prove an algorithm is optimal (meaning of minimal length) if the length is significantly greater than the length of the program that does the deduction in this axiomatic system. The proof is a simple contradiction: suppose the axiomatic system proves program P is optimal. Write a program CONTRADICTION which goes through all the theorems of the axiomatic system until it finds a program which is proved optimal and whose length is greater than the length of CONTRADICTION. Then, run this program. By construction, CONTRADICTION is shorter than this program and yet has the same output.

If you think about what CONTRADICTION is doing, it is generating the code for a completely different program, and then running it. This gives a hint of the nature of the difficulty in finding a minimal description.

But if you are happy with a crude estimate of the complexity, you just write any old code to simulate the physics, and the length of this code is an estimate of the complexity. This is a useful heuristic that makes precise the desire for simple elegant theories.

-

How many bits would it take to define the standard model plus gravity plus massive neutrinos plus dark matter and dark energy? It's an excellent question.

Let's consider the "new minimal standard model", which is a little out of date but still provides a template for the study of this question. The elements of the lagrangian appear in equations 1 through 6, and are assembled together in equation 7. I estimate that it takes about 300 symbols to write down. I count about 40 parameters. Suppose that on average they are known to two significant figures. Assuming 10 ln 2 bits per digit and 128 ln 2 bits per symbol, and we have about (300 x 128 + 40 x 20) ln 2, i.e. about 27000 bits.

Next, how many bits of knowledge do you need to possess in order to be able to interpret those 27000 bits correctly? E.g. what is the shortest set of functions that one could define in Mathematica, such that they correspond to all the basic quantities that make up predictions by the new minimal standard model? I don't think it would be much more than that 27000 bits, if we're careful not to include all the heuristics for computing the function, in the definition of the function. That is, we want to be able to say e.g. what an expectation value is, without necessarily specifying how to compute it.

-