Fermat's principle does not say that the optical path length is always minimal. It only says that all rays take a local minimum^† of the length, i.e. that you can't in first order decrease the path length by making small deviations from the path. For instance, a ray that hits the mirror at an acute angle and comes back at that same angle would not be locally minimal, because you could make the path shorter by having it hit at a slightly steeper angle. What actually happens instead is that the ray exits at the supplementary angle. In this case, changing the point the ray hits can't to first order change anything about the path-length, because any reduced length in the ingoing path would be offset at least as much by increased length in the outgoing path / vice versa.

You might also argue that even a ray bouncing vertically off a mirror is not locally minimal: couldn't you just have it turn around slightly earlier, before it actually hits the mirror? Well, that's actually topologically forbidden because there must be a fixed phase between the ingoing and outgoing ray. You'd need to shorten the path length by an entire wavelength, but again Fermat's principle only considers the locally shortest path, and within the fixed phase it is the shortest path (point in the $U(1)$ group).

^†_{Generally just a local extremum, in fact. Only, maximal optical-path lengths don't tend to occur natually, because you can always increase the path length by introducing small local detours.}

Fermat's principle does not say that the optical path length is always minimal. It only says that all rays take a local minimum^† of the length, i.e. that you can't in first order decrease the path length by making small deviations from the path. For instance, a ray that hits the mirror at an acute angle and comes back at that same angle would not be locally minimal, because you could make the path shorter by having it hit at a slightly steeper angle. What actually happens instead is that the ray exits at the supplementary angle. In this case, changing the point the ray hits can't to first order change anything about the path-length, because any reduced length in the ingoing path would be offset at least as much by increased length in the outgoing path / vice versa.

You might also argue that even a ray bouncing vertically off a mirror is not locally minimal: couldn't you just have it turn around slightly earlier, before it actually hits the mirror? Well, that's actually topologically forbidden because there must be a fixed phase between the ingoing and outgoing ray. You'd need to shorten the path length by an entire wavelength, but again Fermat's principle only considers the locally shortest path, and within the fixed phase it is the shortest path (point in the $U(1)$ group).

^†_{Generally just a local extremum, in fact. Only, maximal optical-path lengths don't tend to occur natually, because you can always increase the path length by introducing small local detours.}

Fermat's principle does not say that the optical path length is always minimal. It only says that all rays take a local minimum^† of the length, i.e. that you can't in first order decrease the path length by making small deviations from the path. For instance, a ray that hits the mirror at an acute angle and comes back at that same angle would not be locally minimal, because you could make the path shorter by having it hit at a slightly steeper angle. What actually happens instead is that the ray exits at the supplementary angle. In this case, changing the point the ray hits can't to first order change anything about the path-length, because any reduced length in the ingoing path would be offset at least as much by increased length in the outgoing path / vice versa.

^†_{Generally just a local extremum, in fact. Only, maximal optical-path lengths don't tend to occur natually, because you can always increase the path length by introducing small local detours.}

Fermat's principle does not say that the optical path length is always minimal. It only says that all rays take a local minimum^† of the length, i.e. that you can't in first order decrease the path length by making small deviations from the path. For instance, a ray that hits the mirror at an acute angle and comes back at that same angle would not be locally minimal, because you could make the path shorter by having it hit at a slightly steeper angle. What actually happens instead is that the ray exits at the supplementary angle. In this case, changing the point the ray hits can't to first order change anything about the path-length, because any reduced length in the ingoing path would be offset at least as much by increased length in the outgoing path / vice versa.

You might also argue that even a ray bouncing vertically off a mirror is not locally minimal: couldn't you just have it turn around slightly earlier, before it actually hits the mirror? Well, that's actually topologically forbidden because there must be a fixed phase between the ingoing and outgoing ray. You'd need to shorten the path length by an entire wavelength, but again Fermat's principle only considers the locally shortest path, and within the fixed phase it is the shortest path (point in the $U(1)$ group).

^†_{Generally just a local extremum, in fact. Only, maximal optical-path lengths don't tend to occur natually, because you can always increase the path length by introducing small local detours.}