Some places I've read flat out say light actually slows down in a medium, some say the speed decresae is just apparent but not real and that the photons still travel at the constant speed of light.
I'm kinda confused.
Light, even in quantum mechanical form, travels as a wave. Photons as individual particles only exist at the end of propagation, when that wave interacts.
That wave always propagates at $c$, the speed of light in a vacuum.
So how does refraction work? As the wave moves through a medium, it intersects with (usually) the electrons, causing them to vibrate. That vibration does not exactly follow the wave: the E field causes a force, which causes an acceleration, which builds to velocity. That motion of charge causes the reradiation of another, weaker, delayed wave. The combination of the original wave and the reradiated one results in the overall wave being a bit delayed. The more material traversed, the more it’s delayed. In that sense, it’s showing a slower velocity: the more material it goes through, the longer it takes to get there.
Note that this is a continuous coherent shaping of a wave. It’s not the scattering of point-like particles like balls in a pachinko machine. This is consistent with what we see: there’s no random scattering of light going through clear optical glass
Photons are massless and they do travel at speed c in vacuum when measured locally.
Though, in a medium, you can read phrases saying that the speed of light decreases. I do understand that you are confused.
If you want to go very basic, then you can say that the photons as they travel in the medium between the atoms and molecules, still travel in vacuum, at speed c. QM is a tricky beast because the photons do interact with the atoms/molecules in the medium. Now there are different approaches as to what kind of interaction this exactly is, that is, scattering (elastic or inelastic), or absorption and re-emission.
Without going into detail which one of those specific interactions is the real one (in reality we do not know), the interaction is there and needs time. The individual photons interact with the atoms/molecules in the medium, and this takes time. Though, the individual photons as they travel inbetween the atoms, they do travel in vacuum at speed c.
Nonetheless, the denser the medium is, the more interactions the photons have to have to propagate, and the more the speed of light slows down.
The simplest picture is that light always travels at the speed of light. But in a material it travels at the speed of light until it hits an atom. It is then absorbed and re-emitted in the same direction, which takes a small amount of time. The more this happens, the slower the effective average speed. The denser the material, the more atoms there are in the way.
It is the wavefront that slows down, and that is why we use classical phrases like the speed of light in a medium slows down.
Depending on the natural frequency of the atom and the frequency of the wave, the emitted photons will change phase when compared to the other, unaffected photons. Therefore, it either falls backward or forward a bit. Since this happens every time the wave hits an atom and there are many atoms in even the smallest piece of material, this has the affect of slowing the wavefront as the wave advances through the substance.
So each individual photon travels at speed c inbetween the atoms/molecules, thought the interaction with the atoms takes time, and this slows down the classical wavefront.
A photon is absorbed by one of the dielectric molecules, so, for a fantastically fleeting moment, it is gone. The absorbing molecule lingers for of the order of 10−15s in its excited state, then emits a new photon. The new photon travels a short distance before being absorbed and re-emitted again, and so the cycle repeats. Each cycle is lossless: the emitted photon has precisely the same energy, momentum and phase as the absorbed one. Unless the material is birefringent, angular momentum is perfectly conserved too. For birefringent mediums, the photon stream exerts a small torque on the medium. Free photons always travel at c, never at any other speed. It is the fact that the energy spends a short time each cycle absorbed, and thus effectively still, that makes the process have a net velocity less than c.
The other answers explain how light appears to slow when it is slowed. But sometimes it isn't, and it was helpful for me to think of the limiting cases for light in water.
Imagine light entering water from a vacuum. Water has a molecular spacing on the order of a nanometer, so from the point of view of a photon with wavelength 1000 nm, the water is a sea of electrons. The photon interacts with the medium as a wave in a dielectric, and the other answers describe how wave-particle duality affects our understanding of the photon's travel.
Now imagine a photon with wavelength 0.01 nm -- an X-ray photon. To this photon, the water is mostly vacuum with some giant H2O particles wiggling around. Since the molecular spacing is much larger than the photon's wavelength, it will interact with the water more as a particle, traveling through the underlying vacuum until it scatters with a single H2O molecule. And we see this in the refractive index of water, which approaches 1 in the nanometer scale (blue line in plot below).