What is the fundamental reason for noise?

Question

I have read that noise is a result of there being "no such thing as a perfect one-way valve". That energy transfer is never perfectly one-way; there will always (at some level) a finite flow of energy from output to input.

I understand this much, but what is the fundamental reason for this? Does it have its roots in thermodynamics?

Edit: The idea of a "one-way valve" is merely a construction in the mind, it doesn't refer to an actual valve, like in a car engine or something. It's just a general way of describing 'what connects two systems'.

Additionally, although I am looking for the fundamental reason for noise, it may be easier to let people know that I am referring specifically to measurement noise.

Answer 1

In statistical mechanics and thermodynamics you are describing systems with an extremely large number of possible variables or degrees of freedom, so describing EXACTLY what happens becomes impossible. Instead, you describe the average. To do this, you consider all physically possible configurations of your system, and say they are all equally probable. You use this ensemble of hypothetical systems to determine the average values of physical properties of your system.

In nearly every case, the average value is dominated by the configurations "near" to your most probable configuration, and fringe cases (like all the atoms in your room being in a cubic cm volume at the same time) are impossibly rare. However, if you have arrangements of your system that are close enough to the most probable, they have a reasonable chance of occurring. So, for example, energy can transfer from a hot system to a cold system just from pure random chance of the collision between molecules in each system. This random movement around the most probable configuration of your system is "noise," and you can apply similar logic to any system described in a statistical manner.

Answer 2

There are different sources of noise, and many different contexts that noise occurs, and when one says there is, "no such thing as a perfect one-way valve", one needs to be more specific about the particular context one is referring to. If one uses a one way valve where the input is a high pressure system, and the valve is isolating the high pressure from the low pressure system (the output side), then the proper context of a valve not performing perfectly is that there is a leakage of high pressure to low pressure and not really that there is low pressure leaking into the high pressure (although there are those who would probably argue its a matter of convention, much like their is an argument about the conventional flow of charge and electrons).

Noise in a thermodynamic system is closely linked to the concept of entropy which relates to the possible configurations a system can be in based on its macroscopic state. If we wanted to think in terms of a leaking valve, there are real mechanical issues such as back flow and the related concept of back pressure.

In any case, if we recognize the idea that molecules are what is flowing in a mechanical system, then there are configurations that are consistent with the macroscopic state of the system where some molecules are on the opposite side of a valve. Through any multitude of mechanisms, there is some small probability that some molecules will find themselves on the wrong side of the valve. Again, this is best understood in terms of entropy. As the disorder increases (entropy increases), energy must be added back into the system to return the entropy to its beginning value. The best way to understand this is by studying the Carnot cycle in thermodynamics.

Noise in information theory (and effectively electronics) is understood best through the works of Claude Shannon and other pioneers. It is important to understand that noise is usually a term associated with unwanted effects.

As far as to the fundamental nature of noise, we could certainly argue that we accept noise as fundamental, but why do we do so? The only real statement we have is the second law of thermodynamics and that is a postulate (or axiom in modern terms). This is something we hold true in principle, and we hold it to be true because it has never been violated, however it also closely related to the central limit theorem which has been proven to be true mathematically.