The speed of light in a medium depends on the (real part of the) refractive index $n$ of the medium. The speed of light in a medium is given by
$$c = c_0/n$$
where $c_0$ is the speed of light in vacuum. Since the refractive index $n$ of the prism depends on the wavelength, we obtain dispersion. However, as soon as the light leaves the prism the refractive index $n_{air}$ must be used. Since $n_{air}\approx 1$ for all wavelengths in the visible range, there exists no dispersion. All wavelength move with the same speed $c = c_0/n_{air}$.
Don't think of the refractive index as a friction. If the index of refraction exceeds one it slows down the speed of light, but no energy is lost -- again, we exclude absorption and concentrate on the real part of the refractive index.
Maybe one more idea to get your head around: Suppose white light enters a medium with "significant" dispersion, but the angle of incidence is $0°$. In this case the different wavelength have different velocities, but the refraction angle is zero. Hence, the rays with different wavelength reach our detector at different times, however, they are overlapping. Thus, the following image sketches helps us to understand the physics
As said before, the colours are not shifted in their position. Nevertheless, in the upper image I shifts the three colours in $y$-direction to explain the "superposition picture", while the lower image shoes the "experienced color":
- At time $t_0=0$ the blue light hits the detector. Hence, we detect "blue".
- At time $t_1 = s_1/c_{green} = n_{green} \cdot s_1/c_0$ the green light ray hits the detector. Hence, we detect the superposition of "blue" and "green". We experience magenta.
- Finally, at time $t_2 = s_2/c_{red} = n_{red}\cdot s_2/c_0$ the red light ray hits the detector. Hence, we detect the superposition of "blue", "green", and "red". We experience white light again.
