What is classical information?

Before I delve into the quantum information science, I have a very basic question on classical information. Please feel free to correct me if any of the following understanding is flawed.

I know and understand that any (real) rational number can be written as some sequence of $$0$$s and $$1$$s. For that, I have the decimal to binary conversion in mind. What do we mean by a piece of classical information? A rational number? Then it is fine.

Now consider a string of heads and tails? Is that also a classical information for a system of coins? Then that too can be written as a string of $$0$$s and $$1$$s by assuming heads correspond to $$1$$s and tails correspond to $$0$$s or vice-versa. Similarly, if we have a string of 8 dice with different outcomes from $$1$$ to $$6$$, for example, $$2,5,5,3,1,4,6,1$$. Is that a classical information for a system of 8 dice? Here too, each number in this string can be expressed as a combination of $$0$$s and $$1$$s by using the decimal to binary conversion.

Now consider a system of $$5$$ coloured balls and an arrangement in a string of sequence red, green, blue, blue, yellow. Is that a piece of classical information of the said system? If yes, can I represent it by a sequence of $$0$$s and $$1$$s, and how?

• What does this have to do with physics? -- And what is the issue of encoding colors in binary to get a bit string? Apr 20, 2020 at 10:26
• ... because Q. information is a subject of physics? Need to know about what classical info. means at first? Apr 20, 2020 at 13:53
• I'm not saying it shouldn't be there ... but I was wondering a bit. With the same logic, you could ask basic calculus questions here ... ;) Apr 20, 2020 at 16:19
• – glS
Apr 21, 2020 at 8:54
• I think nowadays everyone has a pretty decent intuitive idea of what is classical information: we measure it in megabytes (or gigabytes).
– lcv
May 19, 2020 at 7:21

Information is a resolution of uncertainty, as opposed to entropy, which is a measure of uncertainty. Both are measured in bits as a matter of convenience. Thus, for a random event with probabilities of states $$p_i$$ the entropy is given by $$H(p) = -\sum_i p_i\log_2p_i.$$ Once we measured a particular outcome, we say that the uncertainty is reduced to zero, which means that we gained $$H(p)$$ bits of information.

The subject of information is well discussed in the literature dealing with encoding, Shannon's theorem, speech processing etc. One classical reference is Cover&Thomas. Shannon's original paper is quite old, but is still worth reading in the original.

Prefatory Bits

Zeroeth bit: The ability to represent information as a series of ones and zeros is unrelated to whether the information is classical or quantum. Any informational state--classical or quantum--can be written as a series of ones and zeros. A classical computer can do any calculation a quantum computers can do; the classical computer only requires exponentially more memory and time to do so.

First bit: You can represent more than just rational numbers by ones and zeros. For example, $$\sqrt{2}$$ is defined as the larger of the two solutions of the polynomial equation $$x^2 - 2 = 0$$. I can encode the polynomial by its coefficients $$(1, 0, -2)$$ and specify the second root by $$(1, 0, -2, 2)$$. The number $$-\sqrt{2}$$ could then be represented by $$(1, 0, -2, 1)$$. There is a whole set of algebraic numbers that can be specified this way with variable length list of numbers. There is an even larger set of numbers known as the computable numbers that can be specified by the computer program source code that computes them.

Tenth bit: For your colored balls on a string, once you pick a numerical encoding for colors (RGB values, for instance), you can represent a string as a list of numbers representing the colors in the order they appear from left-to-right. This list of numbers is then easily convertable to ones and zeros. This is essentially what bitmap images are.

Classical Information (or: Buckets, Pebbles, and How to Count in Fancy Ways)

So, on to classical information. The primary aspect of classical information that distinguishes it from quantum information is this: If we have complete knowledge of the state of a classical system, then we can predict with certainty what we will see when we observe the system.

Let's take a concrete example. We have an empty bucket. We know it is empty because if it is turned upside-down nothing falls out. Now, we toss three pebbles into the bucket. Even without looking into the bucket, we have complete knowledge of the state of the bucket: it has three pebbles in it. If we observe the state of the bucket by looking inside, we can be certain (barring hallucinations or thieves) that we will see three pebbles inside. We can represent the state of the bucket with the overly fancy notation $$|B_C\rangle = |n\rangle$$, where $$B_C$$ is a label for the classical bucket state and $$n$$ is the number of pebbles in the bucket. Currently, the state of our bucket is $$|B_C\rangle = |3\rangle$$.

Here's an important observation: If we start with two empty buckets and perform the same sequence of operations on them (adding or removing pebbles), then it is guaranteed that we will see the same number of pebbles in each bucket at the end of the algorithm.

A note: even though our example buckets holds discrete pebbles, meaning that $$|3.45\rangle$$ is a nonsensical state, there's nothing stopping us from filling the bucket with water so that we can express continuous quantities (ignoring for now the atomic nature of water and pretending it is a continuous fluid). If we start with an empty bucket, put a known amount water in, and later measure how much water is in the bucket, we can always predict how much we will measure, and two buckets of water prepared identically will always have the same amount of water.

How about some more fancy notation? We want to know if my bucket has four pebbles in it. We can write this query like so: $$\langle 4|$$. Applying this query to a bucket with $$n$$ pebbles is written as $$\left|\langle 4|B_c\rangle\right|^2 = \left|\langle 4|n\rangle\right|^2$$. The result of this operation is $$1$$ if $$n=4$$ and $$0$$ otherwise. That is, $$\left|\langle 4|3\rangle\right|^2 = 0$$ and $$\left|\langle 4|4\rangle\right|^2 = 1$$. In classical information systems in which we have complete knowledge of the state of the system, the only possible answers to this query is $$0$$ or $$1$$.

To summarize:

1. In a classical information system, complete knowledge of the system state enables the prediction of observations with certainty.
2. In a classical information system, identically prepared systems will always result in identical observations.

Quantum Information (or: What Even Is Counting?)

Now, let's upgrade to quantum buckets and quantum pebbles. All of the operations on a classical bucket are still possible. We can add a pebble, and we can remove a pebble (if the bucket is not empty). But, we can do more with my quantum bucket. Not only can we put the bucket in states like $$|0\rangle$$, $$|1\rangle$$, $$|2\rangle$$, $$|3\rangle$$, etc., but our quantum bucket can be prepared in such a way as to exist in this state: $$|B_Q\rangle = \frac{3}{5}|2\rangle + \frac{4}{5}|7\rangle.$$ Now, the proper response to this claim is a good bit of head scratching. Does this bucket have two pebbles inside or seven? Let's apply observation math to this bucket.

If we look for two pebbles, we get \begin{align} \left|\langle 2|B_Q\rangle\right|^2 &= \left|\frac{3}{5}\langle 2|2\rangle + \frac{4}{5}\langle 2|7\rangle\right|^2 \\ &= \left|\frac{3}{5}\langle 2|2\rangle\right|^2 \\ &= \frac{9}{25}. \end{align} If we look for seven pebbles, we get \begin{align} \left|\langle 7|B_Q\rangle\right|^2 &= \left|\frac{3}{5}\langle 7|2\rangle + \frac{4}{5}\langle 7|7\rangle\right|^2 \\ &= \left|\frac{4}{5}\langle 7|7\rangle\right|^2 \\ &= \frac{16}{25}. \end{align} Hmm. Puzzling. Let's just look in the bucket and see what's there. For this bucket, we see seven pebbles. That makes a bit of sense since it had the bigger coefficient. But, remember the second classical observation: identically prepared systems yield identical observations. So, we prepare several thousand buckets to the same state and count the pebbles in each one. Here we find that 36% (9/25) of the buckets have two pebbles and 64% (16/25) have seven pebbles.

So, we have to modify our information observations for quantum systems:

1. In a quantum information system, complete knowledge of the system state enables only probabilistic prediction of observations.
2. In a quantum information system, identically prepared systems will not necessarily result in identical observations.

In quantum systems, complete knowledge of the state of the system ($$|B_Q\rangle$$ above) only allows us to calculate the probabilities of observations--exact probabilities, but still probabilities. For half a century after the beginning of quantum mechanics, it was still up for debate whether, if we observe three quantum pebbles in a quantum bucket, the bucket actually had three pebbles before you looked. Quantum theory and experiment say no. The state $$|B_Q\rangle$$ is the complete description of the state of the bucket before the observation. The bucket actually contained a weird blending of two pebbles and seven pebbles at the same time.

So ... ?

Quantum buckets can be manipulated in many more and weirder ways that classical buckets. For example, this other bucket state $$|B_{Q2}\rangle = \frac{3}{5}|2\rangle - \frac{4}{5}|7\rangle.$$ is somehow a different state than $$|B_Q\rangle$$, even though the observation probabilities work out the same. When there are multiple buckets that can interact, even weirder things can occur like entanglement and interference. This is what allows for speedups in certain computations.

Of course, talking of buckets and pebbles is only a euphemism, so maybe it's time we had the talk.

The rational number example is not a piece of classical information it's just a state of the system just like two coins having two states which can be represented by 01 or 00 ( 0 = heads; 1 = tails). Yes, you can represent any classical system with string of binaries.

Classical information measures the amount of uncertainty present in the system and it depends only on the probability distribution. It is defined by entropy function

$$H(X) = -\sum p(x) log_{2}(p(x))$$

where $$p(x)$$ is probability distribution and sum is over support of $$p(x)$$.

For your last example here is one way of coding the balls:

Red = $$00$$ Blue = $$01$$ Green = $$10$$ Yellow = $$11$$

If the balls are in sequence red, green, blue, blue, yellow then its representation is $$00,10,01,01,11$$. But it has nothing to do with information of the system. Information will depend only on the probability of balls having a particular color.

Broadly speaking, physical information in the context of Classical and Quantum mechanics consists of pure states and mixed states.

This addresses the difference between Quantum pure and mixed states

This explains pure and mixed states in Quantum systems rigorously

First, you would need a binary number that represented all possible colors. So for instance, if you have 5 total possible colors that any individual ball can be then "color" can be represented by a 3-bit binary number. However, a 3-bit binary number can reach up to $$7$$ so the last three possible combinations (representing $$5$$,$$6$$ and $$7$$) of the 3-bit number would be unused (which is fine). Three are unused instead of two because we can use $$0$$ to represent a color. This means, $$111_2$$ ($$7$$), $$110_2$$ ($$6$$) and $$101_2$$ ($$5$$) would never show up.

Next, we would need separate binary numbers to represent the state, or color, of each ball.

If red was $$0$$ then it would be represented by $$000_2$$ and if yellow was $$5$$ then it would be represented by $$101_2$$. Then for two balls with these results we would get:

$$000_2$$ (red)

$$101_2$$ (yellow)

...

and so on if we had more balls and assuming order didn't matter (meaning the balls are identical apart from color). Thus, we'd get a string of binary numbers representing the outcome of your experiment measuring the color of a system of balls.

• In quantum mechanics, superposition states are also pure states. Apr 20, 2020 at 8:14
• I'll amend my answer to clarify this better but unfortunately I don't have time to go into great depth. Apr 20, 2020 at 8:32
• I'd ask this on the Quantum Computing SE if you want a thorough answer Apr 20, 2020 at 9:13
• The statement that "pure states" are "classical information" is plain wrong. Also, mixed states of classical basis states are perfectly classical info. So the whole answer gives a completely wrong conception. Apr 20, 2020 at 10:25
• @NorbertSchuch Yes, you're right. A classical coin flip could be considered a mixed state as just as in spin. I'm not going to deal with getting this all technically correct and lucid so I will reduce my answer to links elsewhere. Unfortunately, that may not be as helpful but at least I can avoid misleading anyone Apr 20, 2020 at 10:33