# Tag Info

94

What you're looking for is Landauer's principle. You should be able to find plenty of information about it now that you know its name, but briefly, there is a thermodynamic limit that says you have to use $k_BT \ln 2$ joules of energy (where $k_B$ is Boltzmann's constant and $T$ is the ambient temperature) every time you erase one bit of computer memory. ...

51

It sounds as though you may be groping for the Bekestein Bound (see Wiki page of this name) from the field of Black Hole Thermodynamics. This bound postulates that the total maximum information storage capacity (in bits) for a spherical region in space $R$ containing total energy $E$ is: $$I\leq \frac{2\,\pi\,R\,E}{\hbar\,c\,\log 2}\tag{1}$$ where $I$ is ...

35

The thing about the speed of light $c$ is that it's not just a number associated with a certain type of particle. While we could talk about the mass of the proton, and there would be no problem assuming non-protons had greater or lesser masses, the value $c$ is an entirely different beast. $c$ is an intrinsic property of spacetime itself, not of the ...

33

Landauer's principle (original paper pdf | doi) expresses a non-zero lower bound on the amount of heat that must be generated by computers. However, this entropy-necessitated heat is dwarfed by the heat generated through ordinary electrical resistance of the circuitry (the same reason light bulbs give off heat).

33

Short Answer The information is contained in the heat given off by erasing the information. Landauer' Principle states that erasing information in a computation, being a thermodynamically irreversible process, must give off heat proportional to the amount of information erased in order to satisfy the second law of thermodynamics. The emitted information is ...

32

Let's suppose you have a Rubik's cube that's made of a small number of atoms at a low temperature, so that you can make moves without any frictional dissipation at all, and let's suppose that the cube is initialised to a random one of its $\sim 2^{65}$ possible states. Now if you want to solve this cube you will have to measure its state. In principle you ...

31

How is the claim "information is indestructible" compatible with "information is lost in entropy"? Let's make things as specific and as simple as possible. Let's forget about quantum physics and unitary dynamics, let's toy with utterly simple reversible cellular automata. Consider a spacetime consisting of a square lattice of cells with a trinary ...

25

In the case of relativity, "information" refers to a signal that enforces causality. That is, if event A causes event B, then some signal must travel from A to B. Otherwise, how would B "know" that A had occurred. Some examples: Light (signal) from a candle (A) hits your eye (B), causing you to see it. Electricity (signal) flows from a connected switch ...

23

The resolution to Maxwell's demon paradox is mostly understood to be through Landauer's principle, and it is one of the most compelling applications of information science to physics. Landauer's principle asserts that erasing information from a physical system will always require performing work, and particularly will require at least $$k_B T \ln(2)$$ of ...

19

Human senses, nearly all, work in a manner and obey Weber–Fetcher law, that response of the sense machinery is logarithm of an input. It is true at least for hearing, but also for eye sensitivity, temperature sense etc. And of course, in areas where it works normally. Because in extreme, there are other processes such as pain, etc. So as in a cause of ...

19

UPDATE: Below I am answering yes to the first question in the post (are the two kinds of entropy the same up to a constant). This has led to some confusion as both Matt and John gave answers saying "the answer is no", however I believe they are referring to the title "Does entropy measure extractable work?". Although the author uses the two interchangeably, ...

18

Assuming infinite precision in measurement, an infinite number of bits can be stored in a single atom. Take the information you want to store, encode it into a string, and then calculate the Gödel number of the string. Call that number n. Then, excite a hydrogen atom to exactly the n${}^{\rm th}$ energy level. In practice, the properties of a real ...

17

In short: information contained in a physical system = the number of yes/no questions you need to get answered to fully specify the system.

17

I don't know anything about the history of the Bel and related measures. Logarithmic scales--whether for audio intensities, Earthquake energies, astronomical brightnesses, etc--have two advantages: You can look at phenomena over a wide ranges of scales with numbers that remain conveniently human-sized all the time. An earthquake you can barely detect and ...

17

I don't know in which context Susskind mentioned this, but he probably meant time evolution is unitary. That means, among other things, that it's reversible, ie no information can ever get lost because you can essentially, starting from any time (time-like slice), run time backwards (theoretically) and compute what happened earlier. If black hole evolution ...

16

Information is a purely mathematical concept, usually a a characteristic of uncertainty (of a probability distribution function), but can be interpreted in different ways. In the simplest form it is introduced in information theory as a difference between uncertainties of two distributions, with uncertainty being the logarithm of a number of possible ...

14

It's just because sounds that the human ear is capable of hearing range over a very large range of amplitudes. If you talked about the power delivered to the ear, rather than the log of the power delivered to the ear, you would need to use numbers like $10^{12}$ to talk about airplane engines. So, rather than deal with that, we use logarithims, so that ...

13

I wouldn't say the ignorance interpretation is a relic of the early days of statistical mechanics. It was first proposed by Edwin Jaynes in 1957 (see http://bayes.wustl.edu/etj/node1.html, papers 9 and 10, and also number 36 for a more detailed version of the argument) and proved controversial up until fairly recently. (Jaynes argued that the ignorance ...

13

Since there are already outstanding technical answers to this question, I think we should add some better philosophical underpinnings for you to explore that might help with gaining a better intuitive feel for what information is. Warren Weaver provided an excellent discussion on information theory in 1949 in his paper entitled "Recent Contributions to The ...

13

Electromagnetic waves travel at the speed of light, and nothing can carry energy or information faster than light. Quantum entanglement doesn't carry information from one particle to another: all you get on one end is a random value from some distribution that has a relationship to a random number somewhere else. They can't be used to transmit information ...

11

In principle, any of the fundamental "forces" could be used to transmit information. In practice, humans are only able to use electromagnetism. And in any case, none of these "forces" travel faster than light. Gravitational waves basically travel as fast, but no faster. Any massive particles (including neutrinos) will travel strictly slower than light. ...

11

Assuming a typical computer with CPU processing power ~1 GHz. It means that it can generate output byte sequence at ~$10^9$ byte/s, which is about ~$10^{-13}$ J/K in terms of von Neumann entropy. Also, the power consumption of a typical CPU is ~100 W, which gives entropy ~0.3 J/K at room temperature. So the (minimum ΔS) / (actual ΔS) ~ $10^{-14}$ This ...

11

What is different here? In some reference frames, your friend guesses the information and acts before you send it and in others, he guesses and acts after you send it. But there is no causality problem since his action is caused by his guess rather than the received information. In all reference frames, the guess precedes the action. Now consider the ...

10

Some info from ASIC world: For example, you processor have 300 mil. transistors, and most of these do some work. But, in order to make for example pure 32-bit add operation you need just about 1000 of them. Others are for caching and passing data back and forth - support functions which are impossible to estimate. So estimations from math side are very hard ...

10

Whether or not neutrinos would be suitable for rapid trading, people have seriously considered their utility for signalling in difficult environments. I read an article a while back about a paper (published in Phys. Lett. B, but I can't access that from here) by Patrick Huber which proposed using neutrinos for through-the-earth communication to submarines as ...

9

I think that the best way to justify the logarithm is that you want entropy to be an extensive quantity -- that is, if you have two non-interacting systems A and B, you want the entropy of the combined system to be $$S_{AB}=S_A+S_B.$$ If the two systems have $N_A,N_B$ states each, then the combined system has $N_AN_B$ states. So to get additivity in the ...

9

Lubos' answer is correct: information is not an observable so does not have fluctuations in the sense that could enter an uncertainty relation. However, there does exist a relationship between 'information' and the uncertainty principle, although not of the type that it seems the OP expects. First of all, note that 'information conservation' could never be ...

9

This is a good question with a lot of deep math and physics behind it (information theory). I will try to give you a casual answer. Signal to noise ratio: First, you should ask yourself what a "signal" is. For example, when you listen to the radio, especially AM radio, you hear the sounds / music / voices just fine even though there is static / noise in ...

Only top voted, non community-wiki answers of a minimum length are eligible