Why doesn't a typical beam splitter cause a photon to decohere? In many experiments in quantum mechanics, a single photon is sent to a mirror which it passes through or bounces off with 50% probability, then the same for some more similar mirrors, and at the end we get interference between the various paths. This is fairly easy to observe in the laboratory.
The interference means there is no which-path information stored anywhere in the mirrors. The mirrors are made of 10^20-something atoms, they aren't necessarily ultra-pure crystals, and they're at room temperature. Nonetheless, they act on the photons as very simple unitary operators. Why is it that the mirrors retain no or very little trace of the photon's path, so that very little decoherence occurs?
In general, how do I look at a physical situation and predict when there will be enough noisy interaction with the environment for a quantum state to decohere?
 A: 
In many experiments in quantum mechanics, a single photon is sent to a mirror which it passes through or bounces off with 50% probability, then the same for some more similar mirrors, and at the end we get interference between the various paths. This is fairly easy to observe in the laboratory.

This experimental fact tells me that the photon has 50% probability to scatter back  each time  and 50% to go through the collective field of the atoms of the mirror.
The elastically scattered photon, by definition keeps its momentum and phases to the universe except for the direction.

Why is it that the mirrors retain no or very little trace of the photon's path, so that very little decoherence occurs?

Your problem then is with the through going photons in a 50% transparent  50% reflective medium.
The elastically scattered ones by definition/solution-of-the-qunatum-mechanical-boundary-condition-problem cannot leave a mark . Classically a ball bouncing off an infinite mass wall does not lose any energy. If there is some diminution in energy of the scattered photon due to momentum conservation, with the whole mirror,  it will be very small, because of the size of the mirror and the "size" of the photon.
The ensemble of the incident photons on the mirror   finds the medium transparent,  because it  does not interact with the electrons forming the molecules and structure of the solid .  It is as if the solid is not there. Why?  Because the energy levels of the molecular, and atomic structure, including vibrational and rotational , do not match the frequency of the wave, ( in this case optical frequency). If they matched it, the medium would not be transparent. Thus the reflected wave and the through going wave keep their original phases and can produce the interference patterns .

Photons pass through the material because they don't have sufficient energy to excite a glass electron to a higher energy level. Physicists sometimes talk about this in terms of band theory, which says energy levels exist together in regions known as energy bands. In between these bands are regions, known as band gaps, where energy levels for electrons don't exist at all. Some materials have larger band gaps than others. Glass is one of those materials, which means its electrons require much more energy before they can skip from one energy band to another and back again. Photons of visible light -- light with wavelengths of 400 to 700 nanometers, corresponding to the colors violet, indigo, blue, green, yellow, orange and red -- simply don't have enough energy to cause this skipping. Consequently, photons of visible light travel through glass instead of being absorbed or reflected, making glass transparent.
In general, how do I look at a physical situation and predict when there will be enough noisy interaction with the environment for a quantum state to decohere?

The phases will be lost if the scattering is inelastic. If inelastic scattering is dominant the beam will decohere, which happens with non reflective surfaces.  Reflection and elastic scattering are a different side of the same coin.had
When one reaches the 10^23 or so number of quantum mechanical entities entering collective interactions the loss of coherence will be dependent on the material and the particular boundary conditions of each problem. For light absorption coefficients etc can characterize the decoherence. For other set ups the individual boundary conditions and the collective emergence of phenomena has to be thought about on a case by case basis ( superconductivity comes to mind).
A: Nobody is answering this question, so I'll take a stab at it. 
Consider the mirror. Suppose you started your experiment by (somehow) putting it in a nearly-exact momentum state, meaning there is a large uncertainty in its position. Now, when you send a photon at it, the photon either bounces off or passes through. If the photon bounces off the mirror, it will change the momentum of the mirror. You could theoretically measure the "which-way" information by measuring the momentum of the mirror after you've done the experiment. In this scenario, there wouldn't be any interference. 
However, you didn't do that. You started the mirror off in a thermal state at room temperature. This state can be considered as a superposition of different momentum states of the mirror1, with a phase associated to each one. If you change the momentum by a small amount, the phase associated to this state in the superposition only changes by a small amount. Now, let $p_\gamma$ and $p_m$ be the original momenta of the photon and the mirror, and let $\Delta p_\gamma$ be the change in the momentum when the photon bounces off the mirror. When you send the photon towards the mirror, the original state $p_m$ (photon passes through) will end up in the same configuration as the original state $p'_m = p_m - \Delta p_\gamma$ (photon bounces off). These two states $p_m$ and $p'_m$ had nearly the same phase before you aimed the photon at the mirror, so they will interfere, and if the phase on these two states are really close, the interference will be nearly perfect. 
Of course, a change in momentum isn't the only way for the mirror to gain which-way information. However, I think what happens when you consider the other ways is that they behave much like this, only not anywhere near as cleanly, so they're harder to work with.
1 Technically, it's a mixed state, i.e., a density matrix, and not a pure state. But the basic idea of the above explanation still holds. 
A: Hopefully a sharper restatement of the question is: what's the difference between a mirror and a photocathode?  Experimentally, in a Mach-Zender interferometer we can fold light paths with a mirror while maintaining coherent interference, but passing either beam into the photocathode of a photodetector destroys interference effects, even for photons that didn't take that path.  In both cases, you have an intimate interaction of a photon with a macroscopic object, so you might naively expect both objects to produce decoherence by irreversibly coupling the photon to the environment.
In either a mirror or a photocathode, incoming photons indeed couple with the free electrons in the material.  In a mirror, these electrons immediately re-emit the photons, crucially without being able to absorb any energy.  In a photocathode, the post-collision electrons absorb enough energy to leave the material surface and go on to couple with the environment (such as a detector).  
In solid-state terms, the difference between these cases is attributed to the photoelectric work function in the material: a mirror reflects light because the photon's energy is insufficient to remove an electron from an atom, so the photon has no choice but to leave unaltered.  
In quantum terms, the big difference is again energy: the phase changes in Schrödinger's equation depend completely on the Hamiltonian--the wave function's total energy.  A surface electron coupled with an incoming photon has negative energy inside a mirror, so the wavefunction for this coupling dies away exponentially, resulting in net zero impact on the photon.  Inside a photocathode, the same electron-photon coupling has positive energy, so it can sustain itself indefinitely, permanently affecting the photon's wavefunction and mixing it with the environment.
If you follow Dr. Shor's answer and imagine situations where the mirror could siphon off some of the photon's energy, for example by recoiling fast enough for the doppler shift to result in a longer-wavelength reflected photon, you have not only built a possible photon detector (by measuring the mirror's recoil), but also clearly have a shorter range for possible interference effects (due to the wavelength shift).  
Interestingly, the same object can act as a mirror at long wavelengths and as a photocathode at short wavelengths, so you can get wavelength-dependent decoherence!
A: Disclaimer: below is per my current understanding of quantum phenomena.
I assume your example is for visible light. In case of photons with low wavelength (X-rays), Compton scattering is a known phenomena - that is example where coherence would be lost - from just one "interaction" with one electron.
Visible light wavelength is ~500nm, whereas atom diameter ~ 500pm (1000 times less). So photon of visible light when "comes" to mirror encounters thousands of atoms even if mirror is tiny in macroscopic terms (atoms are heavy for energy/momentum of photon of visible light, no need for 10^20 for "hit" to be almost elastic). So it is like small ball hitting heavy wall, even if wall is not grounded. Due to conservation of momentum + energy of momentum, "hit" is almost elastic.
Photon is not absorbed and re-emitted, as this way information of momentum is lost. Original photon changes direction, but not so much energy and hence wavelength, and hence interference pattern is seen. As "wave-function" of photon changes, by some definitions I would guess it is considered new photon.
A: To be strict, I think it is “all or nothing.” In the “pure” theory, no mirror pathway information can be “actualized.” This has to do with pathways (path integrals) in Feynman diagrams. There is a real state, which is the real state of the photon at generation, which then “jumps virtual” until this photon is finally “observed” at the observatory screen. In the diagrams, there are “incoming” lines which converge at the “point of photon generation.” Intersection of these incoming lines is the instant when the photon is “real.” In the math of virtuaity, the photon then “jumps to virtuality” and stays completely virtual “through” the mirror (or slit) passage.
While virtual, the photon is probabilistically weighted for being “here” or “there” in virtual space. The probability functions (pdfs) involved are themselves completely deterministically defined (and then deterministically determine the probabilistic “weight” of the photon “having” any allowed virtual state). These pdfs ” propagate “instantaneously” all the way to the observatory screen, given the photon’s initial real state at the instant of generation. This is “entanglement” even in a simple mirror or slit experiment. For some real generated photon state, and some final real impact state, the photon is “in virtual flight” even through the mirrors (or slits). One virtual pathway (“subactualizations” along a particular path from ”real" to “real”) is one Feynman path integral. Another is another Feynman virtual photon path integral (which includes a virtual path through the mirrors or the slits). How can one tell which pathway was taken? One cannot in essence, according to QM. Has the photon taken a physical pathway? No it has no, according to the standard interpretation of QM, if you accept that “real” and “virtual” are two different “worlds”
But I think like you. How could the photon not interact with each mirror (or with the matter of a plate having slits)? It physically must. Afterall, the photon’s speed is a finite c, and impact comes after generation. And especially so, if the photon is replaced with an electron, which has mass, and must gravitationally interact (according to GR) with the mass of the mirrors (or the mass of a plate with slits). Why doesn’t this nonzero gravitational interaction “force decoherence” and “wave collapse” from “virtuality to reality” and the instant of mirror reflection or slit passage?
