From page 35 of "Microelectronics" by Millman Grabel
Mobility $\mu$ decreases with temperature because more carriers are present and these carriers are more energetic at higher temperatures. Each of these facts results in an increased number of collisions and $\mu$ decreases.
Limiting for now the discussion to electrons, my intuition suggests that at a higher temperature (without changing the electric field and the number of carriers), a single electron collides more frequently with the ions and so, on average, its velocity is reduced to zero more often. That causes its drift velocity to be less than it would be at a lower temperature.
I can't get an intuition of why an increased number of electrons (at a constant temperature) causes the mobility to decrease. For a given applied electric field, each electron changes its direction at each collision and, on average, it attains a drift velocity $v_d$. That happens for every electron, that are assumed to be independent from one another. The global drift velocity should still be the same $v_d$. So why the number of electrons affects the mobility (that is, the drift velocity since $v_d = \mu E$)?
What should I imagine for holes?