Example of entropy and information

Question

I was trying to write up a basic, intuitive explanation of entropy, and I wanted to see if this example I made up makes sense.

Imagine a cell membrane separating our two compartments. It can contain either a channel that passes Na+ or K+ indiscriminately, or a channel that only allows K+ to get through. Either way, we allow one of eight ions, only one of which is K+, to pass from left to right:

In either case, the K+ ion might happen to be the one that passes.

At left, if we don't know the ion is K+ because the channel passes anything, this increases entropy by k_B ln 8. (we are going from 1 microstate, entropy 0, to 8 microstates, entropy k_B ln 8).

At center, where only K+ can cross, S = 0 on either side (1 microstate with K+ on the left, and 1 microstate with K+ on the right).

But our result either way might happen to be the same (K+ has passed to the right), except we suppose we have one of two channels for which we never specified the enthalpy. What gives??

The way I'm tempted to interpret this: Using a K+-specific channel lets us know which of the 8 ions is K+ - a number from 1 to 8, or 3 binary bits gives us an entropy of -3k_B ln 2. This is in the information about our ions, and reduces the entropy of the system back to 0.

Now this reading has some odd consequences: for example, the "information" the channel tells us is which ion is K+, but not how many ions are K+. I think this may be valid because entropy is about thermal energy, which can move ions back and forth but not transmute them. Also, suppose it were ten ions - then it takes ln 10 = 3.32 bits of information to reduce the entropy for the center instance back to zero. I think it is fair to say we can know fractional bits of information with fractional bits of negative entropy. In general, I like the symmetry between losing a bit of information by letting an ion cross between two chambers, or getting one back by identifying it as one of 4 among 8 ions.

Question: Is this example valid as a way of thinking about entropy and information?

Answer 1

It is better to think ignorance rather than information and think of "ignorance" as the number of possibilities under current knowledge. To idealize your initial setup, suppose the left box contains $L$ volume elements, and somewhere in those elements we have $n_1$ K+ ions and $n_2$ Na+ ions. The remaining $v=L-n_1-n_2$ elements are empty. Given only this prior knowledge about the system, what is its entropy?

We know the number of ions of each type and the size of the box but we don't know where the ions are in the box. The number of possibilities in this simple case is given by the multinomial factor: $$\Omega(L,n_1,n_2) = \frac{L!}{n_1!n_2! v!}$$ This is our "ignorance", the number of possibilities given what we know about the system. Entropy is its logarithm: $$S = \ln \Omega$$ We can now use this method to calculate before and after entropies. Suppose the yellow ion passes through to a compartment of size $L'$, which let's say was initially empty:

We confirm that the second law applies: $$\ln\Omega_{A'}+\ln\Omega_{B'} > \ln\Omega_A+\ln\Omega_B$$

In all cases the issue is ignorance: based on the information we possess (size of the box, number of ions of each kind) we cannot be certain where each ion sits. We can only give a probability for each possible outcome.