Classical Information Theory vs. Quantum Information Theory I am quite familiar with the basic concepts of information theory (sources, alphabets, simbols, strings, information, Shannon's entropy, noisy channels, Shannon's theorems, etc.). I always thought of information theory as a general meta-theory, applicable to a wide range of subjects and completely independent of any specific context of application. However I have came to understand that nowadays we like to split information theory in two separate branches:

*

*Classical information theory (CIT)

*Quantum information theory (QIT)

And I suspect that we like to do this because these two are quite different from one another, for example in CIT we use Shannon's entropy but in QIT we use Von Neumann's entropy; so I get that CIT and QIT are distinct in a meaningful way, however I have difficulties pin pointing the main differences between them and what are the physical quantum effects that create these differences. I would like a summary of all the most relevant point of disagreement between the two theories.
So my question is: What are the main differences between CIT and QIT? And which quantum effects are responsable for these differences?


This question is a crosspost with Quantum Computing SE. Here is the link to the other post.

 A: The Classical interpretation of the reality is certain, in the sense that it assumes that if we know everything about the state of the system, given the initial state of the system, and the interactions it has with other parts of the world, we can finde the final state of it without any uncertainty. Or to put it simply, The uncertainty in classical view of the world comes from our ignorance.
This is not the case for quantum interpretation. In quantum systems given an initial state and calculating all the interactions, we're still not able to predict the outcome of our experiment. The uncertainty in quantum view of the world comes from our ignorance and the quantum behavior of the nature.
As an example consider a bit, in classical information theory a bit is either 1 or 0. so in the most general case for a bit we have $P_1$ probability of it being 1 and $P_0$ probability of being 0. where $P_0+P_1 =1$. This is different for a qubit (short for quantum bit). For the qubit the state is a "vector" in it's Hilbert space, the Hilbert space of a qubit is two dimensional, which means that any state (vector) in the space can be written as a linear combination of it's eigen vector (the eigen vectors can be seen in eigen value theories but consider just two orthogonal vectors that can map the space completely). So for a qubit the most general case would be:
$$
\vert \text{qubit}\rangle = \alpha \vert 1 \rangle + \beta \vert0 \rangle
$$
where $\alpha^2 +\beta^2 = 1$. This is called the superposition of qubit. Note that it's not 1 or 0 and it's not 1 and 0. It is a completely different state, which can be shown as the two eigen state of the system. Now to measure this we have to use the Born rule, first we calculate the density matrix of this system which is:
$$
\rho = \vert\text{qubit}\rangle\langle\text{qubit}\vert
$$
Then by born rule the probability of finding 0 or 1 would be:
$$
\text{Tr}[\rho \vert0\rangle\langle0\vert] = \beta^2
\text{Tr}[\rho \vert1\rangle\langle1\vert] = \alpha^2
$$
By measuring the system you're observing the qubit not as it was but as it collapses.
So to summerize:
-- classical uncertainty comes from ignorance
-- quantum uncertainty comes from the quantum nature of the reality
This is why there are different types of information theory in general. But entanglement, which is a quantum phenomena plays a role to.
a pair of entangled quantum systems are in general correlated, the information is not stored in one or two of the system but in the correlation between them. And by measuring one, one gains information about the two of them. This is a really simple view for this but I hope it's enough.
