1
$\begingroup$

The Shannon entropy of a probability mass function is $$H(S) = -\sum_s p(s)\log_2 p(s),$$ where the sum runs over all the configurations $s$. If a system $S$ stays half of the time in the configuration $\uparrow$ and the other half in $\downarrow$, in this situation

$$H(S) = -p(\uparrow)\log_2p(\uparrow)-p(\downarrow)\log_2p(\downarrow) = 1$$.

In case of spontaneous symmetry breaking, like in the case of the 2D Ising model, we know that, below the critical temperature, the spin "chooses" a direction to point, so for finite systems, the probability of staying in the direction (for example) $\uparrow$, is greater than $1/2$ and, then, the Shannon entropy becomes smaller. At extremely low temperatures, when the system freezes, the entropy is

$$H^{low}(S) = 0.$$

My question is, are these results correct? After all, if you observe the system long enough, the spins can flip to the other state, so even at very low temperature, the system stays half of the time in the configuration $\uparrow$ and the other half in $\downarrow$, so following this reasoning the entropies of each spin are always $1$.

$\endgroup$
2
  • 1
    $\begingroup$ 1. Why are you looking at a single spin? You should be considering a collection of spins. In fact, the collection of spins should be infinitely large if you want to have a genuine phase transition. 2. Time is not really relevant here. What you want is the Shannon entropy associated to the Gibbs measure, which is not evolving in time. Moreover, even if you were looking at the time evolution of your (infinite) spin configuration, the magnetization density will not change direction (it would - albeit very rarely - for finite systems, but the latter do not undergo genuine phase transitions). $\endgroup$ Mar 25, 2022 at 9:08
  • $\begingroup$ If I consider the entropy of the Gibbs measure the problem remains since the system (for finite system and low temperatures) practically has only two possible states, so the joint entropy of the collection of spin is $1$, as in the case of the single spin, while I expect that should be $0$. There is a similar problem for magnetization, I think: if I perform a Montecarlo simulation of a finite Ising system I have nonzero magnetization for $T<T_c$ while the "analytical" magnetization is always zero for finite system and this does not make much sense for me $\endgroup$
    – AbateFaria
    Mar 25, 2022 at 11:39

1 Answer 1

1
$\begingroup$

I am not sure I understand what's troubling you. For simplicity, let me discuss the two extreme cases $T=0$ and $T=\infty$.

Let us consider a system of $N$ spins.

At $T=\infty$, the spins are independent, identically distributed random variables, each taking values $\pm 1$ with probability $\frac12$. The entropy is then $$ -\sum_{\omega} p(\omega)\log_2 p(\omega) = -2^N \cdot 2^{-N} \log_2 2^{-N} = N\log_2 2 = N, $$ where the sum is over the $2^N$ spin configurations $\omega$, each of which has probability $2^{-N}$ under the Gibbs measure. In particular, the entropy density is equal to $1$.

At $T=0$, the probability measure (with say, periodic boundary condition) is concentrated on the 2 configurations on which the spins all take the same value, each of these two configurations having probability $\frac12$. This yields the entropy $$ -\sum_{\omega} p(\omega)\log_2 p(\omega) = -2 \cdot \frac12 \log_2 \frac12 = 1, $$ and therefore the entropy density (as $N\to\infty$) is equal to $0$.


Concerning your other question in the comments (about the discrepancy between the expected value of the magnetization which is always $0$ (under periodic boundary condition) and the observation that the magnetization is nonzero in your numerical simulations at low temperatures, this is addressed in many questions on this site. For instance, this one. It is related to the fact that typical configurations at low temperatures have magnetization (close to) either $m^*$ or $-m^*$, each with probability $\frac12$; this indeed has expectation $0$. Note that this "problem" disappears in the thermodynamic limit, provided you consider a pure state, as explained in the answer to the question I link to above.

$\endgroup$
2
  • $\begingroup$ Ok now I got it. To clarify, I got confused in the interpretation of Shannon entropy, which should quantify the randomness of a system. My reasoning was that at low temperature, the system is "ordered" in the sense that if I wait that the system "evolves" (given that we can talk of evolution), I expect that the next configurations are (almost) all identical and then the Shannon entropy (not the entropy per spin) should be $0$. Of course, this statement is wrong, since entropy has nothing to do with time evolution but it is just a property of $p(\omega)$. $\endgroup$
    – AbateFaria
    Mar 26, 2022 at 17:41
  • $\begingroup$ I am glad the answer helped. $\endgroup$ Mar 26, 2022 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.