When a photon interacts with an atom three things can happen:
elastic scattering, the photon keeps its energy and changes direction
inelastic scattering, the photon gives part of its energy to the atom and changes direction
absorption, the photon give all its energy to the atom, and the absorbing electron moves to a higher enrgy level as per QM
In your case, when light gets refracted inside glass, it is elastic scattering, it is called Rayleigh scattering, specular reflection. This is the only way that a mirror image is kept, and the photon keeps their energies and phases. And all the photons get coherently scattered.
As photons make their way through glass, the lattice acts like in the double slit experiment. In the double slit experiment, even if you shoot one photon at a time, it will travel as a wave, and parts of the wave will interact with each other. This will create interference. The only constructive interference will be the path that the wavefront will take (bright parts of the screen). All the other directions will not be used because of destructive interference (dark parts of the screen).
Same thing happens in glass with the lattice. The waves go through the lattice's spaces, and the photons' waves will interfere. The destructive interference will cancel out every direction other then the wavefront's direction. Constructive interference will make the wavefront move in one certain direction.
Now virtual absorption means the scattering process. The photons interact with the atoms, but the electrons (around the nuclei) in this case will not absorb them physically, and the electrons will not move to higher energy levels as per QM. In the case of glass this is elastic scattering and because of that, the photons keep their energy and phase, and the electrons (around the nuclei will not gain kinetic energy).
Now you are asking basically about why and how light passing through glass can still make glass hotter. Wiggle the electrons is not correct to say. What is correct to say, is that in the case of glass, all three things happen with the photons, elastic scattering (this makes the image move in the glass), inelastic scattering, and absorption. It is the ratio of these three interactions that is different then in other material. In glass, the ratio of elastic scattering is the highest.
The ratio of inelastic scattering, that makes as you said wiggle, is smaller in glass, but it still works. Now in the case of inelastic scattering, the photons give part of their energy to the atoms and molecules. We have to clarify, that what you are talking about, the wiggle, is called vibration of the molecules. That is what we call temperature of the glass. When photons give part of their energies to the atoms and molecules of the glass, the vibrational energy of the molecules rises, temperature rises.
The ratio of real absorption in glass is smaller too. Some of the photons get really absorbed by the atoms' electrons, and then those electrons get relaxed, and re-emit photons.
You are asking about dispersion, a continuous spectrum contains many different colors or wavelengths, with no gaps. Perfectly white light shined through a prism creates dispersion. This is a continuous spectrum.
The reason you get a speed less then c, is because the way you calculate speed for the wavefront. You are using a straight path as distance, and divide it by the time the wavefront needs to pass through the glass. The individual photons always travel with speed c (when measured locally), because they always travel between the atoms in vacuum. But as the photons get scattered by the atoms of the glass, they change path, because the waves interfere, and the only constructive interference will be the path the photon will go from atom to atom. But that path will not be the same path through the glass as the straight path you calculate speed with.
Here is a picture of a photon making its way out if the Sun. It might take 100000 years for a photon to get through the dense gas. Is the speed of the EM wave that slow? No. The photon travels with speed c (when measured locally) between the atoms. But since it interacts with so many atoms, and as it travels as a wave, parts of the wave interfere with each other, and that changes the path of the photon.
It is a little bit similar in glass, and that is why the wavefront slows down. individual photons still travel at speed c between the atoms.