Your intuition is wrong on this. Consider one-dimensional steady flow, say in the $x$-direction, with a velocity gradient in the $y$-direction. Thus the particles at a given level have average velocity $$\bar{\mathbf u}=(u(y), 0, 0)^T,$$ and fluctuating velocities $${\mathbf u}'=(u', v', w')^T.$$
Let's consider particles that at time $t_0$ are located at $(x,y_0)^T$, which have velocities ${\mathbf u}=(u_0+u', v', w')^T.$ These particles will, on average, travel a distance of the mean free path length $l$ at that velocity, before hitting other particles. The particles will thus have migrated to a different $y$ position, where the average particle velocity will be $$\bar{\mathbf u(y)}=(u(y_0)+(y-y_0)\frac{\partial u}{\partial y}, 0, 0)^T.$$ Notice that the average difference in the $x$-component of the velocity of such particles will therefore be proportional to the mean free path $l$ times an integral $I$ over the distribution of $v'$ and $w'$ velocities which does not matter here: We have $y-y_0=I\,l$. The mean velocity difference for such particles is therefore just $\bar{\Delta u}=I\,l\,(\partial u/\partial y)$.
Since the mean velocity is assumed to stay constant, such particles will have their velocity adjusted to the one at their new $y$-position. Viscous forces correspond to the work required to achieve this. These forces must therefore be proportional to the velocity gradient times the mean free path length.
P.S.: Also see the derivation in the Wikipedia article on viscosity.