# Tag Info

108

What you're looking for is Landauer's principle. You should be able to find plenty of information about it now that you know its name, but briefly, there is a thermodynamic limit that says you have to use $k_BT \ln 2$ joules of energy (where $k_B$ is Boltzmann's constant and $T$ is the ambient temperature) every time you erase one bit of computer memory. ...

52

It sounds as though you may be groping for the Bekestein Bound (see Wiki page of this name) from the field of Black Hole Thermodynamics. This bound postulates that the total maximum information storage capacity (in bits) for a spherical region in space $R$ containing total energy $E$ is: $$I\leq \frac{2\,\pi\,R\,E}{\hbar\,c\,\log 2}\tag{1}$$ where $I$ is ...

39

Short Answer The information is contained in the heat given off by erasing the information. Landauer' Principle states that erasing information in a computation, being a thermodynamically irreversible process, must give off heat proportional to the amount of information erased in order to satisfy the second law of thermodynamics. The emitted information is ...

38

How is the claim "information is indestructible" compatible with "information is lost in entropy"? Let's make things as specific and as simple as possible. Let's forget about quantum physics and unitary dynamics, let's toy with utterly simple reversible cellular automata. Consider a spacetime consisting of a square lattice of cells with a trinary (3-...

36

The thing about the speed of light $c$ is that it's not just a number associated with a certain type of particle. While we could talk about the mass of the proton, and there would be no problem assuming non-protons had greater or lesser masses, the value $c$ is an entirely different beast. $c$ is an intrinsic property of spacetime itself, not of the ...

35

Let's suppose you have a Rubik's cube that's made of a small number of atoms at a low temperature, so that you can make moves without any frictional dissipation at all, and let's suppose that the cube is initialised to a random one of its $\sim 2^{65}$ possible states. Now if you want to solve this cube you will have to measure its state. In principle you ...

33

Landauer's principle (original paper pdf | doi) expresses a non-zero lower bound on the amount of heat that must be generated by computers. However, this entropy-necessitated heat is dwarfed by the heat generated through ordinary electrical resistance of the circuitry (the same reason light bulbs give off heat).

33

Unfortunately there is a loss of information for physical images i.e. images with finite signal to noise ratio per pixel. An out of focus lens acts like a linear transformation, i.e. a matrix between the focused ideal image and the actual image. To reverse that transformation we have to calculate the inverse of the matrix. Depending on the severity of the ...

27

In the case of relativity, "information" refers to a signal that enforces causality. That is, if event A causes event B, then some signal must travel from A to B. Otherwise, how would B "know" that A had occurred? Some examples: Light (signal) from a candle (A) hits your eye (B), causing you to see it. Electricity (signal) flows from a connected switch (A)...

26

The resolution to Maxwell's demon paradox is mostly understood to be through Landauer's principle, and it is one of the most compelling applications of information science to physics. Landauer's principle asserts that erasing information from a physical system will always require performing work, and particularly will require at least $$k_B T \ln(2)$$ of ...

22

In short: information contained in a physical system = the number of yes/no questions you need to get answered to fully specify the system.

21

Human senses, nearly all, work in a manner and obey Weber–Fetcher law, that response of the sense machinery is logarithm of an input. It is true at least for hearing, but also for eye sensitivity, temperature sense etc. And of course, in areas where it works normally. Because in extreme, there are other processes such as pain, etc. So as in a cause of ...

19

UPDATE: Below I am answering yes to the first question in the post (are the two kinds of entropy the same up to a constant). This has led to some confusion as both Matt and John gave answers saying "the answer is no", however I believe they are referring to the title "Does entropy measure extractable work?". Although the author uses the two interchangeably, ...

18

I don't know in which context Susskind mentioned this, but he probably meant time evolution is unitary. That means, among other things, that it's reversible, ie no information can ever get lost because you can essentially, starting from any time (time-like slice), run time backwards (theoretically) and compute what happened earlier. If black hole evolution ...

18

Assuming infinite precision in measurement, an infinite number of bits can be stored in a single atom. Take the information you want to store, encode it into a string, and then calculate the Gödel number of the string. Call that number n. Then, excite a hydrogen atom to exactly the n${}^{\rm th}$ energy level. In practice, the properties of a real ...

17

I don't know anything about the history of the Bel and related measures. Logarithmic scales--whether for audio intensities, Earthquake energies, astronomical brightnesses, etc--have two advantages: You can look at phenomena over a wide ranges of scales with numbers that remain conveniently human-sized all the time. An earthquake you can barely detect and ...

17

I wouldn't say the ignorance interpretation is a relic of the early days of statistical mechanics. It was first proposed by Edwin Jaynes in 1957 (see http://bayes.wustl.edu/etj/node1.html, papers 9 and 10, and also number 36 for a more detailed version of the argument) and proved controversial up until fairly recently. (Jaynes argued that the ignorance ...

16

Information is a purely mathematical concept, usually a a characteristic of uncertainty (of a probability distribution function), but can be interpreted in different ways. In the simplest form it is introduced in information theory as a difference between uncertainties of two distributions, with uncertainty being the logarithm of a number of possible equally-...

14

It's just because sounds that the human ear is capable of hearing range over a very large range of amplitudes. If you talked about the power delivered to the ear, rather than the log of the power delivered to the ear, you would need to use numbers like $10^{12}$ to talk about airplane engines. So, rather than deal with that, we use logarithims, so that ...

14

Since there are already outstanding technical answers to this question, I think we should add some better philosophical underpinnings for you to explore that might help with gaining a better intuitive feel for what information is. Warren Weaver provided an excellent discussion on information theory in 1949 in his paper entitled "Recent Contributions to The ...

13

Electromagnetic waves travel at the speed of light, and nothing can carry energy or information faster than light. Quantum entanglement doesn't carry information from one particle to another: all you get on one end is a random value from some distribution that has a relationship to a random number somewhere else. They can't be used to transmit information ...

12

Assuming a typical computer with CPU processing power ~1 GHz. It means that it can generate output byte sequence at ~$10^9$ byte/s, which is about ~$10^{-13}$ J/K in terms of von Neumann entropy. Also, the power consumption of a typical CPU is ~100 W, which gives entropy ~0.3 J/K at room temperature. So the (minimum ΔS) / (actual ΔS) ~ $10^{-14}$ This ...

11

I think that the best way to justify the logarithm is that you want entropy to be an extensive quantity -- that is, if you have two non-interacting systems A and B, you want the entropy of the combined system to be $$S_{AB}=S_A+S_B.$$ If the two systems have $N_A,N_B$ states each, then the combined system has $N_AN_B$ states. So to get additivity in the ...

11

In principle, any of the fundamental "forces" could be used to transmit information. In practice, humans are only able to use electromagnetism. And in any case, none of these "forces" travel faster than light. Gravitational waves basically travel as fast, but no faster. Any massive particles (including neutrinos) will travel strictly slower than light. ...

11

What is different here? In some reference frames, your friend guesses the information and acts before you send it and in others, he guesses and acts after you send it. But there is no causality problem since his action is caused by his guess rather than the received information. In all reference frames, the guess precedes the action. Now consider the ...

11

Imaginary things can "travel" faster than light A shadow or a light spot can seem to travel faster than light, because it's not a particular physical thing, but a series of separate things, separate physical particles emitted at different time and at different locations. Imagine that you have launched a lot of tiny bots into space with a very accurate ...

10

Some info from ASIC world: For example, you processor have 300 mil. transistors, and most of these do some work. But, in order to make for example pure 32-bit add operation you need just about 1000 of them. Others are for caching and passing data back and forth - support functions which are impossible to estimate. So estimations from math side are very hard ...

10

Whether or not neutrinos would be suitable for rapid trading, people have seriously considered their utility for signalling in difficult environments. I read an article a while back about a paper (published in Phys. Lett. B, but I can't access that from here) by Patrick Huber which proposed using neutrinos for through-the-earth communication to submarines as ...

Only top voted, non community-wiki answers of a minimum length are eligible