3
votes
2answers
130 views

Why the self-information is -log(p(m))?

Why is self-information given by $-\log(p(m))$? Shannon derived a measure of information content called the self-information or "surprisal" of a message $m$: $$I(m) = \log \left( ...
4
votes
2answers
69 views

How can the microstates be measured with zero energy expenditure?

James P. Sethna. Statistical Mechanics. Exercise 5.2: What prevents a Maxwellian demon from using an atom in an unknown state to extract work? The demon must first measure which side of the ...
3
votes
1answer
111 views

Definition of Information in Information Theory

I am not sure in which SE site I have to put this question. But since I have learnt Shannon Entropy in the context of Statistical Physics, I am putting this question here. In the case of Shannon ...
3
votes
1answer
274 views

Should entropy have units and temperature in terms of energy? [duplicate]

I've been thinking about entropy for a while and why it is a confusing concept and many references are filled with varying descriptions of something that is a statistical probability (arrows of time, ...
1
vote
1answer
253 views

An explanation for the Landauer's principle

Has anyone understood the Landauer's principle? What is the current status? In specific, is there a theoretical derivation of the Landauer's Principle?(not the heuristic one based on Salizard's ...
1
vote
1answer
255 views

Do laws of thermodynamics have a place in Theory of Everything? [closed]

I am having a difficulty understanding why second law of thermodynamics is still a valid universally accepted concept. I understand it works on paper for describing isolated heat systems. However, I ...
4
votes
2answers
266 views

Entropy: two explanations for the same quantity?

I studied thermodynamics and I saw the following definition for entropy: $$ \Delta S = \int_1^2 \frac{\text{d}Q}{T} $$ that we use to calculate $\Delta S$ for different types of transformations. In ...
6
votes
2answers
3k views

What is the relationship between Energy, Entropy, and Information?

What is the relationship between Energy, Entropy, and Information? I read this - What Is Energy? Where did it come from? - and the top answer says that 'energy' is an abstract number that is a ...
2
votes
1answer
149 views

Thermodynamics and cross entropy

I am facing with the concept of cross entropy. I would like to know the thermodynamic and statistical meaning of cross entropy (if exists)?
12
votes
4answers
906 views

Ignorance in statistical mechanics

Consider this penny on my desc. It is a particular piece of metal, well described by statistical mechanics, which assigns to it a state, namely the density matrix $\rho_0=\frac{1}{Z}e^{-\beta H}$ ...
6
votes
1answer
321 views

What is the information geometry of 1D Ising model for a complex magnetic field?

Consider the one-dimensional Ising model with constant magnetic field and node-dependent interaction on a finite lattice, given by $$H(\sigma) = -\sum_{i = 1}^N J_i\sigma_i\sigma_{i + 1} - h\sum_{i = ...
21
votes
3answers
257 views

Does entropy measure extractable work?

Entropy has two definitions, which come from two different branches of science: thermodynamics and information theory. Yet, they both are thought to agree. Is it true? Entropy, as seen from ...
6
votes
3answers
1k views

Maxwell's Demon Constant (Information-Energy equivalence)

New Scientist article: Summon a 'demon' to turn information into energy The speed of light c converts between space and time and also appears in e=mc^2. Maxwell's Demon can turn information supplied ...