Why doesn't Rayleigh scattering happen at low altitude in air? An answer I found online was: At high altitude where molecules are far apart, scattered photons can travel without interfering with each other, thus they fill the sky with blue light. But at low altitude, molecules are so close to each other that scattered photons destructively interfere with each other to cancel each other out, that's why we see air as transparent.
However, if they perfectly cancel out, then wouldn't the energy be not conserved?
Another explanation was simply that the effect is only apparent in a very large column of air since the scattering effect is not that noticeable.
So why doesn't Rayleigh scattering happen at low altitude in air?
 A: There is no problem with energy, and an explanation in terms of waves is more intuitive. How else could one easily understand a wavelength dependence?
First there is dispersion, the increase of the index of refraction at short wavelength, which can be understood with the model of Lorentz oscillators.
In a gas there are fluctuations in the number density on short length scales. That means that the index of refraction of cubes with a size of $\lambda^3$ will fluctuate. This gives more scattering at shorter wavelengths, together the $\lambda^{-4}$ dependence.  
I am not sure about the form of the pressure dependence, but I would agree with @planetmaker that your assumption is wrong. Fluctuations are relatively larger at low densities, but the lower the density the less scattering there will be. In the limiting case of the vacuum  there is no blue sky.
A: The assumption is wrong. Rayleigh-scattering does happen independent of altitude. That it happens at sea level, too, becomes clear when you look at the other bank of a large river mouth or a nearby coast over a bay: they loose much contrast and everything is blue-ish; the same effect happens in the mountains looking from one peak to the other.
In the sky there is no other light source, so that only scattered light arrives from there which makes it so nicely blue.
