Entropy of a spin spontuaneous simmetry breaking? The Shannon entropy of a probability mass function is
$$H(S) = -\sum_s p(s)\log_2 p(s),$$
where the sum runs over all the configurations $s$.
If a system $S$ stays half of the time in the configuration $\uparrow$ and the other half in $\downarrow$, in this situation
$$H(S) = -p(\uparrow)\log_2p(\uparrow)-p(\downarrow)\log_2p(\downarrow) = 1$$.
In case of spontaneous symmetry breaking, like in the case of the 2D Ising model, we know that,
below the critical temperature, the spin "chooses" a direction to point, so for finite systems, the probability of staying in the direction (for example) $\uparrow$, is greater than $1/2$ and, then, the Shannon entropy becomes smaller.
At extremely low temperatures, when the system freezes, the entropy is
$$H^{low}(S) = 0.$$
My question is, are these results correct? After all, if you observe the system long enough, the spins can flip to the other state, so even at very low temperature, the system stays half of the time in the configuration $\uparrow$ and the other half in $\downarrow$, so following this reasoning the entropies of each spin are always $1$.
 A: I am not sure I understand what's troubling you.
For simplicity, let me discuss the two extreme cases $T=0$ and $T=\infty$.
Let us consider a system of $N$ spins.
At $T=\infty$, the spins are independent, identically distributed random variables, each taking values $\pm 1$ with probability $\frac12$. The entropy is then
$$
-\sum_{\omega} p(\omega)\log_2 p(\omega) = -2^N \cdot 2^{-N} \log_2 2^{-N} = N\log_2 2 = N,
$$
where the sum is over the $2^N$ spin configurations $\omega$, each of which has probability $2^{-N}$ under the Gibbs measure. In particular, the entropy density is equal to $1$.
At $T=0$, the probability measure (with say, periodic boundary condition) is concentrated on the 2 configurations on which the spins all take the same value, each of these two configurations having probability $\frac12$. This yields the entropy
$$
-\sum_{\omega} p(\omega)\log_2 p(\omega) = -2 \cdot \frac12 \log_2 \frac12 = 1,
$$
and therefore the entropy density (as $N\to\infty$) is equal to $0$.

Concerning your other question in the comments (about the discrepancy between the expected value of the magnetization which is always $0$ (under periodic boundary condition) and the observation that the magnetization is nonzero in your numerical simulations at low temperatures, this is addressed in many questions on this site. For instance, this one. It is related to the fact that typical configurations at low temperatures have magnetization (close to) either $m^*$ or $-m^*$, each with probability $\frac12$; this indeed has expectation $0$. Note that this "problem" disappears in the thermodynamic limit, provided you consider a pure state, as explained in the answer to the question I link to above.
