Strictly speaking, displacement/time is the average velocity, not the average speed. Consider a particle that starts and ends its motion at the same place. Then its average velocity is zero, but its average speed (where speed is the absolute value of velocity) is greater than zero.
Displacement/time is the average velocity with respect to time. This is not necessarily the same as the average velocity with respect to distance. To see the difference, consider a particle that travels at $4$ m/s for a distance of $6$ metres and then at $12$ m/s for a distance of $6$ metres. It has travelled $12$ metres in $2$ seconds so its average velocity with respect to time is $6$ m/s, but its average velocity with respect to distance is $8$ m/s.