The critical point of a general statistical system is a point in the space parameterized by intensive quantities, especially temperature and pressure, at which there exist no boundaries between two different phases of the material even though the boundaries exist at an infinitesimally nearby point. It's the end of a co-existence curve for two phases.
Outside the critical point, the different phases are clumped because there's a correlation in the phase or other properties of nearby atoms or molecules. This correlation persists at some characteristic length scale, the correlation length, which may be visualized as the "thickness" of the boundary between phases. At the critical point, the correlation length becomes infinite so the boundaries are completely fuzzy. Because an infinite correlation length doesn't contain any information about a finite length scale – one cannot use an infinite length as a unit – the theory describing the behavior at the critical point is scale-invariant or, in a somewhat more technical and detailed description, it is a conformal field theory (a theory in which properties only depend on angles). So the critical behavior implies scale invariance. Of course, it is just the scale invariance of the effective description. All the effects may arise from some underlying microscopic physics that does have a characteristic scale; but when we talk about the critical behavior, we study the effects at distances much longer than these microscopic length scales.
For example, the critical point of water is at 374 °C and 218 atmospheres of pressure. The difference between the vapor and liquid water decreases as you go closer to this point and completely disappears at this point. So vapor and liquid are flowing through each other, creating clouds, and you can't really objectively separate them. The material exhibits self-similarity – the clouds are fractals of a sort – which is a general feature of scale-invariant systems. Being scale-invariant is pretty much the same thing as being self-similar. The co-existence of liquid and gas is an example that is pretty easy to imagine. However, there exist many more examples in solid state physics etc. In superconductivity, one finds tricritical points at which the co-existence of three phases terminates, much like the co-existence between two phases terminates at a critical point.
Because the correlation length is infinite, we encounter the truly difficult behavior.