Why are the Lyapunov and Lindeberg Central Limit Theorem conditions often satisfied in the real world?

Question

Some background for the question.

I've been trying to understand why so many things have a Gaussian Distribution. There are a lot of questions about this on StackExchange but none of them were answered in sufficient detail to satisfy me.

First, I know often times people model phenomenal as Gaussian when they are not to make the math easier. I am not asking about these. I am wondering why so many phenomena actually are Gaussian or approximately Gaussian.

Second, people often say a Gaussian satisfies maximal entropy because energy is conserved and is quadratic (E=.5*mv^2). However, this only means that velocity distributions for systems in thermal equilibrium are Gaussian. However, many things besides velocity such as human height are also Gaussian in the real world.

Third, the Central Limit Theorem is often put as an explanation. People claim that the sum of independent random variables tends to have a Gaussian Distribution so long as it satisfies certain conditions. I believe they are referring to the Lyapunov and Lindeberg variants of the Central Limit Theorem.

Which brings me to my actual question: Why are the conditions of the Lyapunov and Lindeberg Central Limit Theorem often satisfied in the real world?

Answer 1

There are many ways of looking at this problem. I think it can be broken up into a few separate but related questions:

1. Why do many things appear approximately normal? For example human height isn't normally distributed, but it still seems well-approximated by a normal distribution.
2. Does the CLT actually apply to most real-world distributions?
3. What is so special about maximum entropy distributions anyway?

I'll try to answer these questions. Let's start with 3. There are different schools of thought in probability; the subjectivist interpretation says that distributions are merely a reflection of our belief or uncertainty in something. If I say I am thinking of an integer between 1 and 10 and ask you to guess what it is, your degree of belief as to what number it is might be, for example, a uniform distribution. Given that you don't know anything else about it, the uniform distribution would be the best choice (it would give you the highest probability of being right). However, you might exploit the fact that humans aren't good random number generators, and so a distribution that weighs 5 or 6 higher than 1 might give you a better chance of winning. The point to all this is that distributions (according to subjectivism) represent your degree of belief, and nothing more.

As it so happens, the distribution with the maximum entropy given your knowledge about something is the one that gives you the highest probability of being right. And if you know nothing else about some measurement/number except that it follows these rules:

• It is a real number.
• It is not bounded from below or above.
• It has an average value.
• It has a certain degree of variance around this average value.

Then the distribution that gives you the highest chance of being right is the normal distribution. A lot of the time, you can even relax some of these assumptions. For example, if it's bounded, but the bounds are much larger than the variance (human height and weight would be good examples here), then the resulting maximum entropy distribution might be a truncated normal distribution, which looks pretty close to a normal distribution.

Which brings us to question 1. Why do so many distributions that aren't normal still appear approximately normal? For example, the Cauchy, Weibull, Student's t, and log-normal distributions all look approximately normal, for certain parameter values.

The best explanation I can think of for this is that they are all the maximum entropy distribution given some constraints, and the constraints are often similar to the constraints that give the normal distribution, so their shapes can wind up being similar to the normal distribution.

Note that just these few distributions together cover a huge range of systems. For example, the CLT says that if you have a bunch of different independent random variables and you sum them, you get something that looks normal. But if instead of adding, you multiply them, you get the log-normal! Which itself looks 'pretty close' to normal, especially if you only had a few sample measurements to go on.

Now finally, question 2. As others have pointed out, the CLT does seem to apply to a lot of real-world problems, but that's actually not that critical. Even if some of the assumptions of the CLT are relaxed a bit, the resulting distributions still come out looking normal. For example, even if the random variables that are summed are not completely independent, the sum can still come out looking normal, as long as there is some degree of randomness or uncertainty in the random variables. This is certainly the case for human height - the genes controlling height all have very inter-related effects. But the result still comes out looking very close to normal.

Answer 2

There are two reasons that come to mind: firstly, many real-world phenomena are collective actions with many, many steps involved. Brownian motion is a good example, where thousands of collisions with a small particle can result in its random jiggles seen in a microscope. The familiar pin-board sorting of balls into a Pascal's-triangle distribution also has rank after rank of identical disturbances (random bounces), and approximates well a Gaussian distribution.

The second reason is more subtle: we make measurements with instruments that we calibrate, and calibration takes out the zero-offset, and the linear-with-independent-variable terms in the errors, but retains the second and higher moments (sigma-squared and such); when we consider only small errors, the error distribution dominated by the lowest nonvanishing moment, the second moment, is one of those things we always assume is Gaussian. The fact is, in the limit of small random errors, when only the second moment matters, the Gaussian distribution result is identical to the 'real' result of ANY other distribution.