Can a single classical particle have any entropy?

Question

recently I have had some exchanges with @Marek regarding entropy of a single classical particle.

I always believed that to define entropy one must have some distribution. In Quantum theory, a single particle can have entropy and I can easily understand that. But I never knew that entropy of a single rigid classical particle is a well defined concept as Marek claimed. I still fail to understand that. One can say that in the classical limit, the entropy of a particle in QT can be defined and that corresponds the entropy of a single classical particle. But I have difficulty accepting that that gives entropy of a single Newtonian particle. In my understanding, If a system has entropy then it also should have some temperature. I don't understand how one would assign any temperature to a single classical particle. I came across a paper where there is a notion of "microscopic entropy". By no means, in my limited understanding, it corresponded to the normal concept of entropy. I am curious to know, what is the right answer.

So, my question is, is it possible to define entropy of a single classical particle?

Answer 1

First of all we must distinguish between two things that are called entropies. There's a microscopic entropy, also called Shannon Entropy, that is a functional over the possible probability distributions you can assign for a given system:

$\displaystyle H[p] = -\sum_{x \in \mathcal{X}}\; p(x) \log(p(x))$

where $\mathcal{X}$ is the set where your variable x takes values. And there's a "macroscopic entropy", that is merely the value of the functional above calculated for a specific family of distributions parametrized by some variable $\theta$:

$S(\theta)=-\sum_{x \in \mathcal{X}}\; p(x|\theta) \log(p(x|\theta))$

Now, what happens in thermodynamics and equilibrium statistical physics is that you have a specific family of distributions to substitute in the first expression: the Gibbs equilibrium distribution:

$p(x | V, T, N) = \frac{1}{Z}e^{-\frac{E(x)}{T}}$

where, as an example, we have as parameters the volume, temperature and number of particles, and E(x) is the energy of the specific configuration x. If you substitute this specific family of distributions on $H[p]$, what you'll have is the thermodynamic equilibrium entropy, and this is what physicists usually call entropy: a state function depending on parameters of the Gibbs distribution (as opposed to a functional that associate a real value for each possible choice of distributions). Now, to find what is the apropriate physical equilibrium for this system when those parameters are allowed to vary, you must maximize this entropy (1).

Now here it's common to make the following distinction: x is a microscopic variable that specify the detailed configuration of the system, and V, T and N are macroscopic parameters. It doesn't need to be so. In the specific case of statistical physics the origin of the distribution function is the fact that there are so many degrees of freedom that it's impossible (and even undesirable) to follow them all, so we are satisfied with a statistical description. Under this assumptions it's natural to expect that the distribution would be over microscopic variables with macroscopic parameters. But this is not the only reason why one would use a distribution function.

You could have other sources of ignorance. As an example, you could have the following problem: suppose we recently discovered a new planet on a solar system where there' 2 more planets. It's position $\vec{x}$ and velocity $\vec{v}$ at a given instant $t = 0$ have been measured within some precision $\sigma_x$ and $\sigma_v$. Let's assume that the sources of possible errors in the measures are additive. Then it's reasonable to assume that we have a gaussian probability distribution for the position of the planet:

$\displaystyle p(\vec{x}(0), \vec{v}(0) | \sigma_x, \sigma_v) =\frac{1}{Z} \exp\left(-\frac{x(0)^2}{2\sigma_x} -\frac{v(0)^2}{2\sigma_v} \right)$

where Z is some normalization constant. Now suppose we want to predict this planets position in the future given the current positions of the other planets and their uncertainties. We would have a distribution:

$\displaystyle p(\vec{x}(t), \vec{v}(t) | \vec{x}_i(0), \vec{v}_i(0), \sigma_{x,i},\sigma_{v,i})= \displaystyle p(\vec{x}(0), \vec{v}(0) | \sigma_x, \sigma_v)\prod_{i=1}^{2}\displaystyle p(\vec{x}_i(0), \vec{v}_i(0) | \sigma_{x,i}\sigma_{v,i}) \times$ $\times p(\vec{x}(t), \vec{v}(t) | \vec{x}(0), \vec{v}(0),\vec{x}_1(0), \vec{v}_1(0), \vec{x}_2(0), \vec{v}_2(0))$

where $p(\vec{x}(t), \vec{v}(t) | \vec{x}(0), \vec{v}(0),\vec{x}_1(0), \vec{v}_1(0), \vec{x}_2(0), \vec{v}_2(0))$ would take Newton's equations of motion into account. Note that there's a small number of particles here: just 3. And the only source of "randomness" is the fact that I don't know the positions and velocities precisely (for a technological reason, not a fundamental one: I have limited telescopes, for example).

I can substitute this distribution in the definition of entropy and calculate an "macroscopic entropy" that depends on the positions, velocities and measurement precisions of the other planets:

$S(x_i, v_i,\sigma_{x,i},\sigma_{v,i}) = - \int d\vec{x} d\vec{v} p(\vec{x}, \vec{v} | t, \vec{x}_i, \vec{v}_i, \sigma_{x,i},\sigma_{v,i}) \log \left[p(\vec{x}, \vec{v} |\vec{x}_i, \vec{v}_i, \sigma_{x,i},\sigma_{v,i})\right]$

What does this entropy means? Something quite close to what thermodynamic entropy means!!! Is the logarithm of the average configuration space volume where I expect to find the given planet in instant t (2)!!! And it's the entropy of a 'single particle'.

There's no problem with that. I can even have situations where I must maximize this entropy! Suppose I don't know the position planet 2, but I do know all three planets have coplanar orbits. There are well defined procedures in information and inference theory that say to me that one way of dealing with this is to find the value of $\vec{x}_2$ that maximizes the entropy, subject to the constraint that all orbits are in the same plane, and then substitute this value in the original distribution. This is often called "principle of maximum ignorance".

There are interpretations of thermodynamics and statistical physics as an instance of this type of inference problem ( please refer to the works of E. T. Jaynes, I'll give a list of references below). In this interpretation there's nothing special on the fact that you have many degrees of freedom besides the fact that this is what makes you ignorant about the details of the system. This ignorance is what brings probabilities, entropies and maximum entropy principles to the table.

Refrasing it a bit, probabilities and entropies are a part of your description when ignorance are built in your model. This ignorace could be a fundamental one - you can't know something about your system; could be a technical one - you could know if you had better instruments; and even, as in the case of statistical physics, a deliberate one - you can know, at least in principle, but you choose to leave detail out cause it isn't relevant in the scale you're interested in. But the details about how you use probabilities, entropies and maximum entropy principles are completely agnostic to what the sources of your ignorance are. They are a tool for dealing with ignorance, no matter the reasons why you are ignorant.

(1) For information-theoretic arguments why we have to maximize entropy in thermodynamics please refer to E. T. Jaynes' famous book "Probability Theory: The Logic of Science" (3) and this series of articles:

Jaynes, E. T., 1957, Information Theory and Statistical Mechanics, Phys. Rev., 106, 620

Jaynes, E. T., 1957, Information Theory and Statistical Mechanics II, Phys. Rev., 108, 171.

Another interesting source:

Caticha, A., 2008, Lectures on Probability, Entropy and Statistical Physics, arxiv:0808.0012

(2) This can be given a rigorous meaning within information theory. For any distribution p(x) let the set $A_\epsilon$ be defined as the smallest set of points with probability greater than $1 - \epsilon$. Then the size of this set must be of order:

$log |A_\epsilon| = S + O(\epsilon)$

For another form of this result see the book "Information Theory" by Cover and Thomas.

(3) Some of Jaynes's rants about quantum theory in this book may appear odd today, but let's excuse him. He committed some errors too. Just focus on the probability theory, information theory and statistical physics stuff which is quite amazing. :)

(4) It seems that dealing with this kinds of problems from Celestial Mechanics was actually one of the first problems that made Laplace interested in probabilities, and apparently he used it in calculations on Celestial Mechanics. The other problem that took his attention towards probability theory was... gambling! Hahaha...

Answer 2

The concept of entropy is very difficult because of the following day-to-day fact: when we have a macroscopic mechanical system, we can look at the system all the time, and know exactly where everything is. In such a situation, we know what each particle is doing at all times, the evolution is deterministic, and the concept of entropy is meaningless.

But the process of looking at particles to find out where they are always produces entropy. To acquire the information about the positions of molecules cannot be done in a way that decreases the entropy of the particles plus the measuring devices. This is an important point, but it can be proven easily from Liouville's theorem. If you start off ignorant of the position of a particle, it occupies some phase space volume. The only way to shrink that volume is to couple trajectories so that you correlate the trajectory of the atoms in a measuring device with the trajectory of the particle. You can do this by adding an interaction Hamiltonian, and this will reduce the phase space volume of the particle given the measuring device trajectory, but the total phase space volume is conserved in the process, so there is uncertainty in the measuring device trajectories which more than compensates for the loss of uncertainty in the position of the particle.

The conservation of phase space probability volume is counterintuitive, because we have intuition that looking classically at particles doesn't disturb them. In fact, if you bounce very weak classical EM radiation from the particles, you can see them without disturbing them. But this is because classical fields do not have a thermal equilibrium--- and when they are near zero over all space, they are infinitely cold. So what you are doing is dumping the entropy of the particles into the infinite zero temperature reservoir provided by the field, and extracting the position from the field during this process.

If you put the field on a lattice, to avoid the classical Rayleigh-Jeans divergence in the thermal equilibrium, then you can define a thermal state for the classical field. If the field is in this thermal state, it gives you no information on the particle positions. If you add a little bit of non-thermal field to measure the particles with, the interaction with the particles dump the phase space volume of the original uncertainty in the particles' positions into the field with a finite entropy per bit acquired, just by Liouville's theorem.

The entropy is a well defined classical quantity, even for a single particle. When you have no information about the particle position, but you know it's energy, the entropy is given by the information theory integral of $\rho\log\rho$. You can extract as much information as you want about the particle by measuring its position more and more accurately, but this process will always dump an equal amount of entropy into the measuring device. All this follows from Liouville's theorem.

This is the reason that entropy is often confusing. When discussing entropy, you need to take into account what you know about the system, much as in quantum mechanics.

Answer 3

Entropy is a concept in thermodynamics and statistical physics but its value only becomes indisputable if one can talk in terms of thermodynamics, too.

To do so in statistical physics, one needs to be in the thermodynamic limit i.e. the number of degrees of freedom must be much greater than one. In fact, we can say that the thermodynamic limit requires the entropy to be much greater than one (times $k_B$, if you insist on SI units).

In the thermodynamic limit, the concept of entropy becomes independent of the chosen ensembles - microcanonical vs canonical etc. - up to corrections that are negligible relatively to the overall entropy (either of them).

A single particle, much like any system, may be assigned the entropy of $\ln(N)$ where $N$ is the number of physically distinct but de facto indistinguishable states in which the particle may be. So if the particle is located in a box and its wave function may be written as a combination of $N$ small wave packets occupying appropriately large volumes, the entropy will be $\ln(N)$.

However, the concept of entropy is simply not a high-precision concept for systems away from the thermodynamic limit. Entropy is not a strict function of the "pure state" of the system; if you want to be precise about the value, it also depends on the exact ensemble of the other microstates that you consider indistinguishable.

If you consider larger systems with $N$ particles, the entropy usually scales like $N$, so each particle contributes something comparable to 1 bit to the entropy - if you equally divide the entropy. However, to calculate the actual coefficients, all the conceivable interactions between the particles etc. matter.

Answer 4

(The question has already been answered by Lubos but perhaps I can elaborate on the quantum aspect.)

With a single classical particle the answer, in general, is 'no' with some exceptions as discussed near the end. And when the number of particles $n \gt 1$ the concept of entropy becomes clear only in the limit that $ n \gg 1$ or the thermodynamic limit (as Lubos mentioned in his answer). With a quantum 'particle' however, the situation is somewhat different.

Consider a qubit, a single spin $1/2$ object, whose state at any given moment is specified by a vector $\vert \psi \rangle$ in a Hilbert space $H$. This is the simplest quantum analog of a classical particle one can think of. In contrast to the classical situation a many-body quantum system can exist either in pure or mixed states. The above specification of the qubit as a state vector is only possible if the state is pure.

A many-body state will not, in general, be describable by a state in a Hilbert space. One must use the more general notion of a density matrix for the correct description. A general density matrix $\rho$ is written in terms of the state-vectors $\vert \psi_s \rangle$ as :

$$ \rho = \sum_s p_s \vert \psi_s \rangle \langle \psi_s \vert $$

where $p_s$ is the "fraction of the ensemble" [reference] in each pure state $\vert \psi_s \rangle $. For instance an ensemble of spin-$1/2$ particles would be described by:

$$ \rho = p_{\uparrow} \vert \psi_{\uparrow} \rangle \langle \psi_{\uparrow} \vert + p_{\downarrow} \vert \psi_{\downarrow} \rangle \langle \psi_{\downarrow} \vert $$

where $p_{\uparrow}$ ($p_{\downarrow}$) is the fraction of particles with spin up (down). Now, on the face of it, it seems one should be able to use this definition for a single particle. It is important to keep in mind that though one can a priori assign probabilities to the two states of a single particle, experimentally these probability amplitudes can be confirmed only by repeating a measurement of the particle's spin many times over. The result of a single measurement is a single number signifying which eigenstate the particle collapsed into due to the measurement process. To obtain a probability one must take the average over many such numbers, which requires more than one measurement.

In this case the entropy loses meaning as the measure of the lack of information external observers have about a system. The "system" is no longer a qubit with a given state or density matrix. Instead it is a qubit which has undergone repeated cycles of the measurement process consisting of state preparation, time evolution and finally collapse to an eigenstate.

The question boils down to the fact that probability amplitudes can only be obtained experimentally by generating an ensemble of observations. A single system cannot be assigned an entropy. A system consisting of an observational ensemble for qubits - either with a single or few qubit and many measurements or many qubits and few measurements - can be assigned an entropy.

---

Extra material The classical notion of Gibbs-Shannon entropy can be extended to the quantum version called the von-Neumann entropy:

$$ S_{cl} = -\sum_k p_k \ln(p_k) \Rightarrow S_{qm} = - Tr\left[\rho \ln(\rho)\right] $$

---

First Edit @Marek makes some pertinent observations in the comments, which I feel are important enough to be included in this edit.

"In contrast to the classical situation a many-body quantum system can exist either in pure or mixed states." what contrast? In classical physics it is also either pure (a single point in the phase space, which is the analogue of the Hilbert space in the classical physics) or a mixture of a pure states which can be described by a measure on the phase space. Everything is completely the same.

The key difference lies in the different effect of measurement on a classical and quantum process. In the absence of a unified description of measurement spanning all domains, we are left with two notions of measurement. In the quantum mechanical case an irreducible error is present in any measurement. This is not the case for a classical mechanical system where measurements can be done to an arbitrary accuracy. Of course, in a unified description of measurement, a classical system would utimately be understood in terms of many-body quantum systems subject to the effects of decoherence.

"A single system cannot be assigned an entropy" -- of course it can be assigned an entropy. Just because you can't measure everything at once doesn't mean it doesn't make sense to define it :)

I'm not talking about "everything" but only of the very definite concept of "probability". Probability is a statistical notion. You cannot do statistics with one data point.

Because if you'd said so, you might as well throw whole quantum theory out of the window, which ultimately doesn't allow you to talk about anything else than probabilities and for those you need multiple measurement as well...

Well yes, of course, you need multiple measurements for quantum probabilities. That is the essence of my argument.

Answer 5

Given a probability distribution, $$S=-k\sum_i p_i \ln p_i.$$

Unless the probability distribution is of the form $e^{-\beta \left( H - \mu N \right)}/Z$, or reasonably close to it, we can't really define the temperature of the particle.

I should also add that if the probability distribution is a Dirac delta function, the entropy goes to zero (at least if we introduce an ultraviolet regulator...).

Answer 6

If the space is unbounded a single particle will have an infinite number of positions. It’s velocity can be any value between zero and c if you believe relativity in that situation, which is problematic. If you know the position relative to some frame of reference, a dubious notion physically because a single particle’s position must be defined against some other physically defined frame...well let’s pretend there is an abstract position based on a ghost consciousness, a dubious idea. Then the entropy is 0. If you forget the position or didn’t originally know it then the entropy is infinite I believe. If it is in a finite space then its entropy is not infinite. An infinite amount of information is required to specify a position in an infinite space.

Don’t believe me. I am not a physicist

Answer 7

The entropy of a single, classical particle is given by noting that the Boltzmann-Gibbs distribution maximizes the entropy over all distributions on phase space, i.e. it is the equilibrium distribution. The derivation works for $N$ particles, and defines temperature as $\beta=\frac{1}{k_B T}$. For $N=1$, this gives an entropy of

$$\frac{k_B}{2}\left(1+ \log\left(\frac{2\pi m L^2}{\beta h^2}\right)\right)$$

for a particle of mass $m$ in a 1d box of length $L$, where $h$ is Planck's constant.

Answer 8

To give a short answer, yes, it is perfectly possible, via $$ mc^2 = S T_P, $$ where $T_P$ is the Planck Temperature.

In my understanding, If a system has entropy then it also should have some temperature.

Indeed; and there is no contradiction: Temperature is a field sourced by entropy. Just like gravitational potential is a field sourced by mass. viz. $$ T(r)=\frac{\hbar c}{k_B^2}\frac{S}{r} $$ (in which I have ignored retarded potentials, etc. For an exact answer you must solve a wave equation). For a thorough discussion of this see here.