Mirror surface flatness is specified as a multiple of $\lambda$, where $\lambda$ is the wavelength at which the surface flatness was tested, and is typically $\sim 632\ nm$. A $4\lambda$ surface flatness beam might not work so well in certain applications with a $632\ nm$ beam. Would the effect of surface flatness increase or decrease when using the same mirror with a long wavelength beam (eg. hundreds of $\mu m$)?
Most lenses and mirrors are carefully designed and manufactured to a shape that captures light that left a point and brings it back to another point, as in this example. This is a camera with lenses, but telescope mirrors and bathroom mirrors do a similar job.
The image is from http://camera-wiki.org/wiki/Lens
If the shape is wrong, some rays will go to the wrong place and the image will be blurred instead of a sharp point.
Lens and mirror design is done by ray tracing. Given the position and angle that an incoming ray strikes a lens or mirror, it is possible to calculate the angle at which it leaves. And therefore the position and angle at the next lens or mirror.
Lens design software can calculate light from multiple points, at multiple wavelengths. For zoom lenses, it must consider multiple lens separations. It is a difficult job to choose shapes that bring all of these to sharp focus.
But light is only approximately described by ray tracing. Light is a wave. You can't bring it to a perfect point focus. You can't make the focal spot smaller than a wavelength. The wave properties of light are calculated as the effect of diffraction in lens design.
So if lenses and mirrors make the rays come to an almost perfect spot that is about the size of the spot from diffraction, that is good enough. The system is said to be diffraction limited, which is as good as you can get.
Lens design software can also calculate how the focal spot is affected if a surface radius is slightly wrong. Typically if the error is big enough that the surface is misplaced by a wavelength of light or so, the rays are misdirected enough to create a focal spot that is bigger than diffraction limited. The software can calculate the exact sensitivity of the focal spot size to errors in each surface.
For visible light, if requires tight tolerances to manufacture an object where the surface shape is correct within a wavelength.
$CO_2$ lasers operate at $10.6 \mu m$ in the far infrared, about $40$ times the wavelength of visible light. Tolerances are considerably looser there, because diffraction limited spots are $40$ times bigger.