How can my window not scramble the image of my yard? How can an image pass through a window if the atoms in the glass randomly emit photons in any direction?
I've read that glass is transparent because the atoms don't readily adsorb visible light, so it passes right through. 
But then how can a glass lens refract anything if it's not even interacting? Yet magnifying glasses burn ants!
I've also read that refraction has to do with the difference of velocity of the wave in different mediums, but I thought that had to do with adsorption also.
Maybe it's because a small number of photons get adsorbed and that holds the whole wave back some how?
Or is this just another one of those places where classical intuition breaks and you have to go mathy/qm?
 A: Very reasonable question. I will try to answer it in an intuitive way.
If you have a scattering medium, photons are reflected in random directions; but when you have a refractive medium, something else happens. The photon is not absorbed and re-emitted: instead, the photon interacts with the electrons in the medium, and since these electrons are somewhat bound to the atoms, a displacement (due to the E field of the photon) results in a restoring force. We can measure the degree of displacement or polarization, and express it in terms of the dielectric constant of the material. If there is a lot of polarization, the dielectric constant is high. And the refractive index can be shown mathematically to scale with the square root of the dielectric constant.
What happens then is that the bound electrons move "behind in phase" with the photon, which results in a phase shift of the EM wave (the original phase of the photon which lost a bit of its amplitude, and a lagging phase of the electron). The photon didn't get destroyed - it got delayed, but maintained its direction.
Now when you have an interface between two materials of different refractive index, and light is incident at an angle to that interface, then a greater phase difference builds up between adjacent beams, which is why light is refracted; but if all the light is traveling through the same medium of constant refractive index, all beams will refract by the same amount and remain parallel.
Which is why you can see through a window.
But if you use ground glass, the surface is no longer flat but has been modified to change the direction - and this results in the image behind the glass becoming fuzzy. The same principle exists in the glass used in many showers (original image from Victoria Elizabeth Barnes's posts on bathroom remodeling:

You can clearly see how the bottom half of the window blurs the image of the trees outside. In the article they call this "pebbled" glass.
