The classic macroscopic variables one typically measures for an ideal gas are $P$, $V$, $T$, $n$, - pressure, volume, temperature, and amount, respectively. I am curious what the corresponding variables are for analogous system I'll call a 'bitgas', and the relationship between the infodynamics and thermodynamics.

A 'bitgas' for the purposes of this question is a string over the alphabet $\{0,1\}$ which is written into the state of a localized physical system. That is, there is a closed 3 dim'l boundary around the system, with finite volume.

As an example, let's take a solid state hard drive $H$ whose capacity $C = 15 \, mol*bit$, or ~1 yottabyte. Suppose the volume is $V= 66.8 cm^{3}$. If Moore's law holds, such a device may be commonplace by 2040. The hard drive is in a room at $300Kel$, and we may or may not hook up a cable $I/O$ (a USB cable, or SATA + power) which can transfer data and/or power.

Let $x$ be a bitstring representing the state of the hard drive. $I/O$ can act on $x$ in one of three ways i) $swap_{ij}$, apply a transposition $(i\,j)$ swapping the bits at position $i,j$ ii) $write_{i}(y)$ where $y\in\{0,1\}$ and the bit at position $i$ is $y$ after the operation iii) $read_i$ transfers the bit $y_i$ out of position $i$.

In this analogy, the hard drive atoms, mechanics, and enclosure represent the classical "container", and the $1's$ which are written in the hard drive are the "gas atoms". In the classical scenario, the gas is the thing with macroscopic thermodynamic properties like pressure and temperature. Here, the hard drive does of course have a temperature and take up volume, but it is a solid. Changing the temperature below a certain critical temperature $T_c$ at which the hard drive melts or burns shouldn't affect $x$.

Define the following variables for the bitgas $H$:

  • $K$ = information content, the Kolmogorov complexity $K(x)$
  • $C$ = capacity of $H$
  • $T$ = temperature of $H$
  • $n_1$ = number of ones in $x$

$n_0 = C - n_1$ is the number of zeros.

Suppose $n_0=n_1=r$, so that there are as many 0's as 1's, and restrict to the case where we only allow the operation $swap$.

For any compression algorithm, we know that some strings will be incompressible, and have large information content so that $K\approx C$. Other strings, such as $x_r=0^{r} 1^{r}$ have low information content.

$n_1/(n_0+n_1)=r/C=1/2$ is constant in this example. However, if we very slowly heat up the hard drive close to its failure temperature $T_c$, we expect errors to occur and bits to start flipping, which may change $n_1/C$.

If we initialize the hard drive to an initial state $x_r$, it would seem $K_0=K(x_r)$ is small and constant while $T<<T_c$, but as $T$ approaches the critical temperature $K$ begins to increase until it reaches $\approx C$.

When $K\approx C$, we will have put about $15*N_{A}*k_{B}*300Kel \approx 37kJ$ into the bitgas.

One could imagine holding a candle at, say, one corner of the hard drive. If that side is all zeros, it will begin to become corrupted. This 'corrupted' portion would contain a lot of information about where the flame was held.

A less extreme example would be to put the hard drive on a hot plate, and slowly increase the temperature until bits start flipping.

It seems that $K$ depends on $T$, and I am wondering what the relationship is, exactly. In other words, what is $\displaystyle \frac{\partial K}{\partial T}$ when $C$ is held constant?

I'll point out that $K$ does depend on $n_1$. When $n_1=C$, all the bits are 1, which is a highly compressible state, so that $K \approx \log(C)$.

  • $\begingroup$ I am not sure how the bitgas is different from Ising model... $\endgroup$ Jul 6, 2020 at 16:35
  • 1
    $\begingroup$ @Vadim The Ising model has a nearest-neighbor interaction, and the two states are at different energies. Neither of these are necessarily true here. $\endgroup$ Jul 6, 2020 at 17:17
  • $\begingroup$ Isinf is not necessarily nearest neighbor, and the two states have the same energy, if there are no magnetic field. So you the question is about non-interacting spins? Aka Stat. Mech. 101? $\endgroup$ Jul 6, 2020 at 18:26
  • 1
    $\begingroup$ Aren't the only two parameters in the Ising Hamiltonian the interaction strength $J_{ij}$ and the field strength $h$? If you turn them both off, the Hamiltonian is zero. $\endgroup$ Jul 6, 2020 at 18:28
  • $\begingroup$ I don't think it is true that $K$ depends on $T$. I would have said $K$ is a function of the integral of $T$ over time. This is not a mere technical distinction. Provided the error rate is greater than zero then I will have $K \approx C$, provided I wait long enough, regardless of what my error rate originally was. $\endgroup$ Jul 7, 2020 at 15:53

1 Answer 1


If the energy level of the 'bitgas' does not depend on the number of bits in state 0 and state 1, then all microstates have the same energy level, and the system is an example of the microcanonical ensemble.

The thermodynamical equilibrium of this system is the macrostate where all microstates have the same probability. This state has entropy S = N*log(2), where N is the number of bits, which you call the capacity C.

Note that for the microcanonical ensemble the temperature is not a relevant quantity. The temperature quantifies how many more microstates become accessible when energy is transferred to the system from the surroundings. But if energy cannot be transferred between the system and its surroundings because the total energy of the system cannot vary, then the temperature is irrelevant.

If your system is initially frozen in a specific state $x_r$ and there is an energy barrier associated with changing state (flipping or swapping spins), then the problem becomes an example of non-equilibrium thermodynamics. The rate with which the system will approach the high-entropy equilibrium from its low-energy initial state will depend on the temperature of the surroundings. It is important to note that for any temperature larger then zero, the system will eventually reach its equilibrium, the question is only how long it takes before it happens.

  • $\begingroup$ For this system, I am more interested in $K$, the complexity, not $S$, the entropy which as you say is $S=C\log(2)$. Note that, assuming the answer to this question (physics.stackexchange.com/questions/563838/…) is accurate, it doesn't require any energy to permute bits around, which will change $K$. If we leave a hard drive at room temperature, we would be very disappointed to find its content changed. However, in your final statement you claim that it does. Roughly how long would this take? Longer than the lifetime of the universe? $\endgroup$ Jul 7, 2020 at 15:55
  • $\begingroup$ The Kolmogorov complexity is very hard to quantify, since it depends on the programming language chosen etc. But since the thermal equilibrium is an even distribution over all possible microstates there will be no consistent smart way of programming the microstate a random timepoint regardless of language. The complexity K will, as you say, be something like the capacity C. $\endgroup$ Jul 7, 2020 at 18:02
  • 1
    $\begingroup$ Even if it does not cost energy to flip spin around (that is, the total energy of the system is the same before and after the flip) then there will still be an energy barrier associated with the flip. Otherwise, thermal noise would disrupt the state immediately. The rate with which the system approaches the thermal equilibrium could perhaps be approximated by the Arrhenius equation. In that case, the rate will decrease exponentially with inverse temperature so for temperatures that are low compared to the activation energy the times might very well be the lifetime of the universe. $\endgroup$ Jul 7, 2020 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.