This question is motivated by this one.

Suppose $l$ is the minimum measurable unit of length. What is entropy of a spinless particle contained in this interval?

We know that entropy of a two-level system depends on the probabilities of the respective levels, if the probability of the state 0 is $p_0$, then the entropy (in natural units) is:

$$S= -\sum_{i=0}^1 p_i \ln p_i = -p_0 \ln p_0 - (1 - p_0) \ln (1 - p_0)$$

So if $p_0=1/2$ then $S=\ln 2\: \mathrm{nat}$, equal to $1\: \mathrm{bit}$. A particle which has the maximum in the middle has entropy of $1\: \mathrm{bit}$ (it is equally likely to be measured to the right and to the left of the middle).

Since we cannot measure intervals smaller than $l$, we cannot make guesses about where the maximum of the probability for the particle is located. As such, if we assume that the particle is equally likely have the maximum of the probability in any point on the interval $x\in[0,l]$, the total entropy becomes $$S=\int_0^l \frac{-(1-\frac xl) \ln (1- \frac xl)-\frac xl \ln (\frac xl)}{l} \, dx=\int_0^1 -(1-x) \ln (1-x)-x \ln (x) \, dx=\frac12$$

An entropy of a similar particle contained in a square area with side $l$ will be twice more, that is $1\: \mathrm{nat}$.

Now if we assume that $l=2l_p$ where $l_p$ is the Planck length, we arrive that such spinless particle has entropy of $1\: \mathrm{nat}$ per 4 square Planck length or $1/4\: \mathrm{nat}$ for one square Planck length.

Thus from the only assumption that double Planck length is the minimum measurable interval, and double Planck length squared is expected to contain 1 particle on average we arrive at the standard value of the Black Hole entropy in nats:

$$S=\frac{A}{4l_p^2}=\frac14 A_p$$

Where $A_p$ is the area in Planck units.

Sometimes I encountered a claim that the fundamental unit of information is 1 bit. From the above considerations it follows that possibly the fundamental unit is 1/2 (or 1 or 1/4) nat.


Note that the distance of $2l_p$ between two particles is natural if we assume that the particles are planckons, whose radius is Planck length $l_p$. As such, the Black Hole can be viewed as a spherical shell consisting of one layer of planckons.

  • $\begingroup$ I'm very confused. If $x$ is a probability and $l$ is a length, how can it make sense to say $x\in [0,l]$? $\endgroup$ – Nathaniel Feb 7 '14 at 1:48
  • $\begingroup$ @Nathaniel in the first example it is probability, in the next example it is the coordinate and the probability of finding particle in l is x/l while for finding particle in 0 it is 1-x/l. If we take l=1 then x is probability. $\endgroup$ – Anixx Feb 7 '14 at 1:52
  • $\begingroup$ I've edited to remove the ambiguity. Please roll back if I got it wrong. $\endgroup$ – Nathaniel Feb 7 '14 at 1:55
  • $\begingroup$ But then, if $x$ is a position coordinate, what sense should I make out of the integrand $-(1-x)\log(1-x) - x\log(x)$? $\endgroup$ – Nathaniel Feb 7 '14 at 1:57
  • 1
    $\begingroup$ Ok, now I get it. You're assuming that a particle confined to such a distance becomes a two-state system because you can't make measurements in the middle of the interval, or something like that, right? $\endgroup$ – Nathaniel Feb 7 '14 at 2:14

The calculation is interesting (I never noticed before that it comes out to $1/2\:\mathrm{nat}$!). Perhaps it does indeed have an interpretation in terms of black holes, but I think it doesn't work in exactly the way you say, because you're actually calculating a conditional entropy rather than an entropy.

Let's look at your calculation purely in terms of probability theory. Let $X$ be a continuous random variable taking values in $[0,1]$ (I've normalised by $l$ for simplicity), and let $M$ (for "measurement") be a discrete random variable taking values in $\{0,1\}$.

Here, $X$ stands not for the particle's position but just for the peak of the wavefunction. Your assumptions correspond to $(i)$ the marginal for $X$ is uniformly distributed over $[0,1]$, and $(ii)$ the conditional probability $p(M=1\mid X=x) = x$. (You state assumption $(i)$ explicitly in the question, and use $(ii)$ implicitly in writing the second equation in your post.) The joint distribution is therefore given by $$p(M=i,X=x) = p(M=i\mid X=x)p(X=x) = \left\{ \begin{array}{ll} xdx &\text{if $i=1$}\\ (1-x)dx & \text{if $i=0$,}\end{array} \right.$$

For a pair of jointly distributed random variables $A$ and $B$, the marginal distribution for $A$ is given by $p(A=a) = \sum_b p(A=a,B=b)$. In this this becomes an integral: $$p(M=1) = \int_0^1 p(M=1, X=x) = \int_0^1 x dx = 1/2.$$

Now, the Shannon entropy of $M$ conditioned on $X$ having some particular value $x$ is given by $-\sum_{i\in\{0,1\}}p(M=i\mid X=x) = -(1-x)\log (1-x) - x\log x$. The conditional entropy is defined as $H(A|B) = \sum_b p(B=b)H(A\mid B=b)$, which in this case becomes an integral: $$ H(M|X) = -\int_0^1 dx((1-x)\log (1-x) + x\log x), $$ which evaluates to $1/2\:\mathrm{nat}$ as you say.

However, the entropy we use in physics is the von Neumann entropy, which corresponds not to a conditional entropy but to the marginal entropy of the measurement (assuming we're measuring in an eigenbasis.) In your system this is given by $$H(M) = -\sum_i p(M=i)\log p(M=i) = -\log(1/2) = 1\:\mathrm{bit}.$$

So (again assuming this is a measurement in an eigenbasis, which it will be if the entropy is maximised, since all bases are eigenbases in that case) the von Neumann entropy according to your assumptions is $1\:\mathrm{bit}$, not $1/2\:\mathrm{nat}$. A particle confined to a square of length $l$ will have a von Neumann entropy of $\log(4) = 2\:\mathrm{bits}$, since it can be measured at any one of the four corners.

  • $\begingroup$ No. I never did an assumption that the particle has uniformly distributed probability density over [0,1]. This is not my assumption! My assumption is that the particle's distribution X has the maximum. If the scale was not close to Plank's, we could assume some point where the maximum is and find the von Neuman entropy of such distribution. But my point is that the Planck scale makes it impossible to locate the maximum of the wave function even statistically BUT the wave function HAS the maximum (as opposed to just flat wave function). $\endgroup$ – Anixx Feb 7 '14 at 3:06
  • $\begingroup$ As you can see from the linked question, this formula arose from the discussion about Knightian uncertainty (uncertainty about probability distribution). $\endgroup$ – Anixx Feb 7 '14 at 3:11
  • $\begingroup$ At the scale of usual quantum mechanics we can assume the probability density (wave function) to be exactly defined. My point is that at Planck scale you cannot assume probability density to be known exactly. $\endgroup$ – Anixx Feb 7 '14 at 3:16
  • $\begingroup$ Sorry, I should have used a different symbol instead of $X$. I know you didn't assume the particle's position is uniformly distributed. Rather, you're assuming there exists some random variable (the position of the maximum of the wavefunction, which I foolishly denoted $X$) that is correlated with $M$ in the way I said. The kind of reasoning I present here is just the Bayesian way of dealing with Knightian uncertainty. If you do things this way, Dempster-Shafer theory becomes unnecessary. $\endgroup$ – Nathaniel Feb 7 '14 at 3:51
  • $\begingroup$ Just to clarify: in Bayesian probability theory, a random variable can stand for a lack of knowledge as well as a stochastic type of uncertainty. In this case the (marginal) probability distribution over $X$ represents your uncertainty about the peak of the wavefunction (which has a real value, but it can't be known), whereas the conditional probability $p(M=i|X=x)$ represents the (quantum) probability of measuring $M=i$ if $X$ were somehow known. In Bayesian theory there is no problem with combining these two different uncertainties into a single joint distribution. $\endgroup$ – Nathaniel Feb 7 '14 at 3:56

You are modelling a two state system for a black hole. The Bekenstein bound really applies with large $N$ states. Witten worked out how a BTZ black hole in 3-dim would have quantum states that converge to a Bekenstein result as $N~\rightarrow~\infty$.


  • $\begingroup$ Off topic : How could you get 10.9k reputations, while having just about the same medails (1 golden, 10 silver and 24 bronze) as me ? You should have much more medails, since you have 8.5 times my reputations! I'm just trying to understand how this system is working. $\endgroup$ – Cham Dec 9 '18 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.