17
$\begingroup$

Noise processes in engineering and physics are frequently assumed to be Gaussian processes. This allows use of convenient analytical techniques. The question then arises as to why natural processes are Gaussian. In particular I'd like to understand why electrical Johnson noise is a Gaussian process.


Possible line of reasoning

One line I've found is that the Ornstein-Uhlenbeck process is Gaussian and satisfies the Fokker Planck equation. This could suggest that any physical process obeying the Fokker Planck equation is an Ornstein-Uhlenbeck process, and therefore a Gaussian process. However, this still leaves the question of why Johnson noise obeys the Fokker Planck equation.

An obvious related question is whether or not Johnson noise is an Ornstein-Uhlenbeck process.

$\endgroup$
12
  • 1
    $\begingroup$ The argument I remember is from the Central Limit Theorem in the form that for large numbers of events, the Poisson distribution tends to match the gaussian distribution. $\endgroup$
    – The Photon
    Jun 17 '15 at 17:53
  • $\begingroup$ Please let my wildly gesticulating giant foam hand throw the terms "uncorrelated emitters" and "central limit theorem" into the fray. It's a very good question, though. $\endgroup$
    – CuriousOne
    Jun 17 '15 at 17:53
  • $\begingroup$ @ThePhoton Gaussian distributed at any particular point in time is not the same as being a Gaussian process. At least, I don't think so. I understand that the central limit theorem explains why so many things in physics are Gaussian; any variable which is a sum of many other variables is always Gaussian, but again (I think) that's not the same as being a Gaussian process. $\endgroup$
    – DanielSank
    Jun 17 '15 at 18:01
  • $\begingroup$ From wiki, a gaussian process is a process for which "for which any finite linear combination of samples has a joint Gaussian distribution". Since one sample is a finite linear combination of samples, this means that a single sample from a gaussian process has a gaussian distribution. (but doesn't mean the two concepts are absolutely identical) $\endgroup$
    – The Photon
    Jun 17 '15 at 18:04
  • $\begingroup$ @ThePhoton It's the converse that I think isn't always true. $\endgroup$
    – DanielSank
    Jun 17 '15 at 18:06
14
+300
$\begingroup$

There are several ways I can interpret the question, so my main focus is going to be on the autocorrelation of an Ornstein-Uhlenbeck (O-U) process. So what is an O-U process and how is it different from regular Brownian diffusion?

Brownian diffusion

The stochastic differential equation (SDE) for Brownian diffusion of a particle can be written as $$\mathrm{d}x_t = \mathrm{d}W_t$$ where $x$ is displacement and $\mathrm{d}W$ a stochastic process such that is we integrate both sides, $x_T - x_0 = \int_0^T\mathrm{d}W_t = \mathcal{N}(0, T)$, the Gaussian distribution with mean $0$ and variance $T$. A bit more physics-y notation would be $$\frac{\mathrm{d}x}{\mathrm{d}t} = \eta$$ where $\eta$ is a Gaussian random variable.

So you can see in your mind's eye that you as you keep adding these small random displacements to random directions, you're going to end up with diffusion. Another way to describe diffusion, and more common to physicists, is a partial differential equation (the Fokker-Planck equation), where you write the probability distribution of the particle as a function of time and position. Yet another way to write this is as a Wiener path integral (the transformation between these representations go through the Feynman-Kac equation).

Ornstein-Uhlenbeck processes

Finally, moving to the O-U equation: $$\mathrm{d}x_t = -x_t\mathrm{d}t + \mathrm{d}W_t$$ or if you prefer $$\frac{\mathrm{d}x}{\mathrm{d}t} = -x + \eta$$ How does the new term change the behaviour? Instead of diffusing to infinity, you are constraining the particle with a linear force. In other words, you have a particle in a harmonic potential, and some noise added to the position. The solution is only a change of variables away: $x_t = e^{-t}y_t$, so by Ito's lemma we have: $$e^{-t}\mathrm{d}y_t - e^{-t}y_t\mathrm{d}t = \mathrm{d}x_t = -e^{-t}y_t\mathrm{d}t + \mathrm{d}W_t$$ so $$y_T-y_0 = \int_0^Te^s\mathrm{d}W_s$$ that is $$x_T = x_0e^{-T} + \int_0^Te^{(s-T)}\mathrm{d}W_s = \mathcal{N}\left(x_0e^{-T}, \frac{1}{2}(1-e^{-2T})\right)$$ where the last step just follows from substituting to the definition of expectation value and variance.

So what do we see? We see that our observation $x_T$ at time $T$ depends on where we last saw $x$. But that this dependence (autocorrelation) dies out exponentially. Indeed, after an infinite amount of time, we have $x_T = \mathcal{N}(0,1)$. Note that we did not get $x_T = \mathcal{N}(0,T)$, but rather that our Gaussian is constrained in size and will get to be that size at an exponential speed (as the effect of the last observation dies out).

Johnson noise

Ok, so you have a resistor in series with an inductor over some voltage and you write $$L\frac{\mathrm{d}I}{\mathrm{d}t} = -RI + V$$ Look familiar? Suppose the voltage is nearly zero, but as the charge carriers themselves undergo diffusion, this voltage is actually stochastic. That's an O-U process. Now, suppose you measure the current at time 0, and then again at time $T$. In what sense are these Gaussian and independent? As discussed in the previous section, if you wait long enough, each measurement is going to be independent from one another and they are both going to sample the same, finite distribution (as opposed to infinitely spreading distribution we saw in diffusion). This Gaussianity and its autocorrelation is what I believe you were after. Finally, let's turn to something I don't think you were after, but may think as interesting nonetheless.

Microscopic noise from the charge carriers

Now why should $V$ be stochastic, and why should it be Gaussian? This is a longer discussion and I am only going to very qualitatively and briefly go over it. Suppose we have some simple system: A particle in a potential and a large number of harmonic oscillators (suggestively called "the heat bath") coupled to it, so that we might write the Hamiltonian (a bit simplified from the Caldeira Leggett model) as $$\mathcal{H} = \frac{p^2}{2m} + V(q) + \sum_i \left(\frac{p_i^2}{2m_i} + \frac{1}{2}m_i\omega_i^2x_i^2\right) + q\sum_ix_i$$

Turns out that this can actually be turned into a generalized Langevin equation of the form $$\frac{\mathrm{d}^2q}{\mathrm{d}t^2} = -\frac{\mathrm{d}V}{\mathrm{d}q}-\int_0^T\mathrm{d}t\frac{\mathrm{d}q}{\mathrm{d}t}\xi(T-t) + R(t)$$ where the memory kernel $\xi(T-t)$ and the "random force" $R(t)$ are both some functions of the stuff in the Hamiltonian. What is remarkable is that the random force, while deterministic, behaves exactly as if it were a "real" Gaussian thermal force (fluctuation-dissipation theorem, its correlation with the other quantities etc). So, we're just going to say that it is random and in this way integrate out some degrees of freedom that we don't care about.

This can be done more rigorously for basically arbitrary systems describing some phase space evolution by using Mori-Zwanzig theory. There you take a projection operator, which projects onto the subspace that contains the degrees of freedom of interest, and the other degrees of freedom then behave like a thermal bath. Many books make this out to be something very complicated, but it's really just matrix (or really, operator) algebra.

Point being that if you have a system and have perfect knowledge of the entire phase space, if you drop out some details, those details will in many ways act as if there was a Gaussian thermal bath pushing the system around.

$\endgroup$
12
  • 1
    $\begingroup$ Is there a reasonably digestible source where a physicist can learn how to read and use stochastic differential equations? $\endgroup$
    – DanielSank
    Jun 17 '15 at 22:31
  • $\begingroup$ @DanielSank The mathematics kind or the physics kind? For the latter, van Kampen's Stochastic Processes in Physics and Chemistry is probably your best bet. It's a terrific book: Works both as an introduction to the subject and as a reference. For the introductory mathematics-y stuff (stuff with the integrals properly defined and with "proper" notation), Oksendal's Stochastic Differential Equations. Finance books also have a lot of SDEs, and as they are in an applied context, you may find those fun as well. $\endgroup$
    – alarge
    Jun 17 '15 at 22:41
  • $\begingroup$ This may be a helpful reference. $\endgroup$
    – DanielSank
    Jun 17 '15 at 22:59
  • $\begingroup$ I want to accept this but I also want to make sure I understand it first. I think the basic reasoning is this: 1) The OU process is Gaussian. 2) A one-pole circuit has the same equation of motion as the OU process (assuming the source voltage is a $dW$, whatever that is). 3) The voltage noise from a resistor is a $dW$ because of the Caldeira-Leggett model. $\endgroup$
    – DanielSank
    Jun 24 '15 at 2:57
  • $\begingroup$ @DanielSank Correct. I wish to, however, emphasize that the distribution you sample from the OU process is iid (which I think you meant to imply with "Gaussian") only if you do it "very slowly": The relaxation time depends on the coefficients L and R. $\endgroup$
    – alarge
    Jun 24 '15 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.