I'm wondering if the information lost by rearranging the letters of a book is measurable as a difference in its initial and final mass.

Choose a long, random string over an alphabet, say $\{0,1\}$, of length $N$. It should be random in the sense that it is incompressible. You might also choose a big book, at random, and compress it.

Once you have the book, or have written down the string in a book, measure the books' mass $m_0$.

Convert the letters into a standard alphabet by using, say, the ASCII encoding scheme. The letters should be more or less distributed uniformly, unlike English which has a rank-frequency distribution for the letters. Rearrange the letters into the complete works of Shakespeare, or as much literature as you possibly can. Then apply the encoding to get a bitstring. This process can be represented by a 0/1 permutation matrix $\sigma_1$ which acts on the bitstring.

Finally, move all the 0's to the left, and 1's to the right. This can be represented as another permutation matrix $\sigma_2$. Measure the mass of the book to get $m_2$.

It appears that the information content of the book at the beginning is $S_0=N$ bits. The information content $S_1$ of the complete works of Shakespeare is around 1.98MB (less than really, size of zipped text file). The last state is very compressible, and $S_2 \approx 2\log_2(N/2)$.

Suppose $N$ is large, say Avagadros' number $N=N_{A}=6.02214076*10^{23}$, more than a zetta and less than a yotta. Then $\triangle S = S_0 - S_2 \approx N_A$. If 1 bit represents about $10^{-23} J/K$, then at $300K$ the information lost corresponds to $~20.1$ picograms.

I suppose the lost information is carried away by the matrices $\sigma_1$, $\sigma_2$ if no one watches or records the rearrangement as it occurs. Is that correct?

  • 3
    $\begingroup$ Related: physics.stackexchange.com/questions/263197/… $\endgroup$ – Rococo Jun 28 '20 at 21:32
  • 2
    $\begingroup$ I've removed a number of comments that were attempting to answer the question and/or responses to them. Please keep in mind that comments should be used for suggesting improvements and requesting clarification on the question, not for answering. $\endgroup$ – David Z Jun 28 '20 at 22:15
  • 2
    $\begingroup$ I've removed a number of comments that were about comment moderation. Please use comments to improve the posts to which they are attached. We can discuss moderation policies on Physics Meta or in Physics Chat. $\endgroup$ – rob Jun 29 '20 at 0:12
  • $\begingroup$ I am not sire what the word lighter refers to in the headline - the question seems to deal with information rather than mass or energy. Unless lighter is used in figurative sense ;° $\endgroup$ – Roger Vadim Jul 6 '20 at 12:39
  • $\begingroup$ I am using it in the literal sense. I am asking, if you rearrange the letters, does the mass change? It's along the sames lines as "Is a hot cup of coffee heavier than a cold cup?". $\endgroup$ – Jackson Walters Jul 6 '20 at 12:56

Summary: Your question is tantamount to asking whether mass-energy equivalence has been extended to mass-energy-information (or mass-energy-entropy). As far as I can see, essentially no one in the physics community (outside of a handful of outliers) accepts such an extension. So the answer to your question seems to be, as far it's known, no. I discuss this below, and also explain why entropy and energy are qualitatively different, even though they are both included in free energy expressions.

Anders Sandberg raises the objection that rearranging the letters destroys semantic but not Shannon information. This can be addressed by posing a somewhat different question:

Let's compare two otherwise identical books. Does a book in which the first half are all zeros, and the last half are all ones, have a different mass from an otherwise identical book in which the ones and zeros are randomly arranged? The latter would have a higher Shannon entropy, since it requires more information to describe the sequence of numbers in the latter book than the former.

Having said this, I believe Anders is correct: Even if the books did have different Shannon entropies, their masses would not differ because of this.

I'll make two related arguments, one based on thermodynamics, and another based on the nature of science generally.

The thermodynamic argument: Mass-energy equivalence applies to, well, energy. Energy is not the same as free energy. Free energies consist of an energy term minus an entropy term (of the form TS). One of the great utilities of a free energy is that it enables us to determine the ability of a system to do work. The more the entropy of the system can increase during a process, the more work (everything else being equal) can be obtained from the system.

So you might ask: Doesn't that mean a lower entropy system has more energy than a higher entropy system? The answer is no. Entropy does not contribute to a system's energy. It is, instead, a measure of the quality of a system's energy—specifically, of how useful (or useless) a system's energy is for doing work.

Sean Carroll has a nice discussion of this on his Preposterous Universe blog:

Thus energy and entropy are qualitatively different things. Hence it would require a significant expansion of the concept of mass-energy equivalence to include entropy as a form of energy.

And, as far as I can tell, such an equivalence has not been established. I say this because mass-energy equivalence and entropy-information equivalence are very important in physics. Hence, if mass-entropy (or mass-information) equivalence (which is really what you're asking about) had also been established, this would be a well-known result (because it would connect mass-energy equivalence and entropy-information equivalence!).

Instead, in checking both Google and Google Scholar, I was only able to find a few papers about mass-entropy/mass-information equivalence, including this one:


Vopson, M.M., 2019. The mass-energy-information equivalence principle. AIP Advances, 9(9), p.095206.

In reading the abstract, and examining the citations, it is clear that mass-energy-information equivalence is, currently, highly speculative.

Hence while is may be possible that the book with lower Shannon entropy (and thus higher free energy) would weigh more, there doesn't appear to be even a small school of physicists that currently accept this.

  • $\begingroup$ “ This can be addressed by posing a somewhat different question”. To be clear, that is exactly the question I asked. The reference you provide looks great, and is essentially the question - what is the result of this experiment? Entropy is indeed quantitative, not only qualitative, and the point I’m making here is that there is both a qualitative and quantitative difference between two states of this system - the state $0^k1^{N-k}$ and a long random string (empty hard drive v. full, and small Kolmogorov complexity vs. large), and that this difference is measurable. $\endgroup$ – Jackson Walters Jul 10 '20 at 8:46
  • $\begingroup$ Right, or mass-energy-information. It seems like there is a lot of discussion to be had to clarify the entropy vs. information dichotomy, and form a consensus. I’m not sure the beliefs of the community are what determine facts about the universe - I was under the impression that experiments are. No (at present) is a perfect fine answer. The question was deliberately asked to avoid speculation, and instead move towards “if we don’t currently know this, how far off are we from having the technology required to run this experiment? 2040, 2060, never?” or “is this the right experiment to run?” $\endgroup$ – Jackson Walters Jul 10 '20 at 17:34
  • $\begingroup$ "I’m not sure the beliefs of the community are what determine facts about the universe - I was under the impression that experiments are." First, I never said "believe", I said "accept". And, second, what the community accepts is determied by a complex framework of both experimental and theoretical results. And one of the best ways to assess the extent to which reliable evidence (both experimental and theoretical) exists that some result X holds, in physics, is to assess the extent to which that result is accepted within the physics community. That's how science works. $\endgroup$ – theorist Jul 10 '20 at 17:41
  • $\begingroup$ Ah, okay that information was deleted so it was difficult to read - that's a good distinction. Right, physics is not mathematics, and the community does have to decide whether the results of an experiment are valid, as at the end of the day mathematicians have to decide if a proof written in natural language is valid (until all the theorem proving software gets good enough in 20-blah-blah). So then, are there any experiments planned to test this hypothesis, or is it too vague to be tested? I suppose I'll read past (or even just) the abstract of the paper you linked. $\endgroup$ – Jackson Walters Jul 10 '20 at 17:43
  • $\begingroup$ Yes, that paper would probably your best starting point point for asking about planned experiments. I'd also recomment looking up the papers that cited that paper, to see what they had to say about it. $\endgroup$ – theorist Jul 10 '20 at 17:48

Rearranging letters in a book destroys semantic information, the information the text gives a reader who can understand it. But it does not change the information in the Shannon or thermodynamic sense, as distinguishable states.

When I read a book I am exploiting pre-existing correlations between my brain and the text, so that reading a certain set of symbols triggers some mental representations. The meaning of the text resides in this mutual information rather than the text itself - a different set of symbols might tell me the "same" message in a different language (or even a permutation). But since the meaning is not in the book it has no effect on the mass or any other physical property.

The information that matters thermodynamically is how many book microstates corresponds to the same macrostate. Rearranging the ink a bit is a minuscule change compared to all the degrees of freedom in the paper molecules that do not matter for the message. Still, a very low-entropy state (all bits zero) would have a slightly different Gibbs free energy $E-TS$ from a high-entropy state (bits randomly zero or one). But it has nothing to do with how much meaning there is in the book. It also does not affect the mass: the stress-energy tensor $T_{ij}$ in general relativity is (as far as I can understand relativistic thermodynamics) independent of the entropy currents.

  • $\begingroup$ Certainly the Kolmogorov complexity of a random string is larger than that of the string $0^{k}1^{N−k}$. Doesn't this imply that the information changes in that sense? In Mathematica, "ByteCount /@ Compress /@ Permute[Table[If[i <= n/2, 0, 1], {i, 1, n}], SymmetricGroup[n]]" gives 72 when n=1,2,3,4, {72,80} when n=5, 80 when n=6,7,8, {80,88} when n=9,10,11,12, and then my computer runs out of memory. For longer strings, we can randomly sample permutations. At n=30, we see the strings split into at least 4 complexity classes of compressed lengths 80,88,96,104. $\endgroup$ – Jackson Walters Jun 29 '20 at 14:19
  • $\begingroup$ If Kolmogorov complexity had measurable effects (e.g. on mass) we could calculate it empirically, calculating something uncomputable by Turing machines. That that could be done would be a very bold claim. $\endgroup$ – Anders Sandberg Jun 29 '20 at 16:33
  • $\begingroup$ As far as I know, the standard model does not predict the masses for any of the known fundamental particles. Could you elaborate on what you mean by calculate it empirically? $\endgroup$ – Jackson Walters Jun 29 '20 at 16:47
  • $\begingroup$ You seemed to imply that the Kolmogorov complexity is the relevant measure. But were it to have an empirically measurable effect, such as mass, then you could calculate it exactly by making a book and measuring the mass change for different texts. This would mean you had a physical method to compute a Turing-uncomputable function. While not a priori ruled out, it is a bit too radical for most computer scientists to believe in. $\endgroup$ – Anders Sandberg Jun 30 '20 at 10:48
  • $\begingroup$ Ah, sorry, I misread your comment. I find it a bit surprising that, on the one hand, we have a mathematical quantity $K(x)$, objects in the world which should have some value for $K(x)$, we're able to calculate good upper bounds for this quantity to the point where we are fairly confident that we're close to $K(x)$, and yet this quantity has no empirically measurable effects, or relation to the SI units. Two large hard drives, viewed as a black boxes, are seemingly physically indistinguishable despite one being empty and the other full? That seems radical. $\endgroup$ – Jackson Walters Jun 30 '20 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.