# All Questions

52 questions
Filter by
Sorted by
Tagged with
23 views

### Proof of continuous Shannon entropy

In Jaynes' Probability Theory: The Logic of Science, there is a quick derivation of continuous entropy in chapter 12. He does so by taking the discrete definition of entropy and combines it with ...
29 views

### Fisher Information in Statistical Mechanics

I am studying the canonical ensemble and it seems to me there is an analogy between derivatives of the partition function, which can extract energy momenta for the system and Fisher score /information....
25 views

### Maximizing mutual information with Restricted Boltzmann Machines and Monte-Carlo sampling

So I've been reading through Koch-Janusz and Ringel's, "Mutual Information, Neural Networks, and the Renormalization Group" (check it out here). I'm currently trying to reimplement some results from ...
59 views

### Is physical entropy opposite to information entropy? [duplicate]

I am currently taking a class on Decision Theory where I was introduced to the notion of "entropy" in an information theoretic setting while we were studying decision trees. I was given the following ...
84 views

### E. T. Jaynes' subjectivism vs measurement of distributions

In his paper, E. T. Jaynes argues that entropy is a measure of our ignorance about a system. As such, the probability distribution of states $\{p_k\}$ has to be chosen in the most unbiased way, thus ...
107 views

### What is the physical interpretation of the action integral, without the stationary action principle?

I'm still wondering about the physical interpretation of the action integral of some mechanical system (classical theory here, to simplify things): \tag{1} A = \int_{t_1}^{t_2} L(q, \, ...
58 views

### Is it necessary to assume that equivalent microstates cannot get transformed into inequivalent microstates to derive the Landauer principle?

The Landauer erasure principle states that to erase a bit of information from a system, the entropy of the environment will be increased by at least $k_B\log2$, or equivalently, it costs at least an ...
89 views

44 views

### How Fast Can (Large) Semiclassical Black Holes Evaporate?

In "Entropy in Black Hole Pair Production" (arXiv:gr-qc/9306023), Strominger et al. notes The issue of whether (1.2) can be taken literally has bearing on the vexing question of what happens to ...
252 views

### Renormalisation and the Fisher-Rao metric

The renormalisation group (I'm talking about classical, statistical physics here, I'm not familiar with field theory too much) can be though of as a flux in a space of possible Hamiltonians for a ...
4k views

### Does entropy depend on the observer?

Entropy as it is explained on this site is a Lorentz invariant. But, we can define it as a measure of information hidden from an observer in a physical system. In that sense, is entropy a relative ...
144 views

### Why does a liter water contain more information than all the information on the whole internet? [closed]

In this (probably well known) picture you can see that a litre water contains much more information than all the information on the whole internet. Now I understand that the total number of possible ...
87 views

### When my body is decomposed into all it's elementary particles, do these particles possess more information than my whole body?

If the elementary particles out of which my body is built up are all disconnected, do these particles contain more information than when they were part of my body? In other words, is the number of (...
329 views

### How is “a bit of information” defined in physics?

I keep encountering the phrase a bit of information but I don't know precisely what this means. I have an intuition of a system that can be in 1 of 2 possible states (in the classical realm that is) ...
252 views

### How to compute entropy of networks? (Boltzmann microstates and Shannon entropy) [closed]

I also asked in SO here a few days ago, thought it may be also interesting for physics-related answers. I would like to model a network as a system. A particular topology (configuration of edges ...
176 views

156 views

### How can “information” be a useful physical quantity given that its value is model-dependent?

From @Humble's answer to "What is information?": Information contained in a physical system = the number of yes/no questions you need to get answered to fully specify the system. That is, however, ...
328 views

### What is the black hole information paradox really? [closed]

Preliminaries What is the black hole information paradox really? Is it a sophisticated way to ponder and debate the existence of an operator on the boundary that can tease out the interior of a ...
667 views

### Books on entropy [closed]

What books introduce entropy in a intuitive, elementary way (at most, for a person with undergraduate physics studies)? The book should not necessarily introduce entropy in relation only to ...
122 views

### Does conservation of information mean that the direction of causality is arbitrary? [duplicate]

If it is the case that the information content of the universe is conserved, and the past can be constructed from a complete knowledge of the future just as easily as vice versa, then is there any ...
162 views

### What are some good books on foundations of Thermodynamics [duplicate]

I am not looking for some introductory texts, graduate text, etc. I am looking for something that addresses to foundational problems in thermodynamics and statistical mechanics.
112 views

### Reference for statistical mechanics from information theoretic view

I am interested in knowing if some one here knows book/notes for statistical mechanics from the information theoretic viewpoint.
5k views

### Is there a thermodynamic limit on how efficiently you can solve a Rubik's cube?

Suppose I build a machine which will be given Rubik's cubes that have been scrambled to one of the $\sim 2^{65}$ possible positions of the cube, chosen uniformly at random. Is it possible for the ...