The formula you wanted to use gives you the magnitude of the average velocity, not the average speed.
To get the magnitude of the average velocity, you take the total displacement (which is a vector!), divide by the total time, and find the magnitude of that vector. What you get is:
$$\text{Magnitude of Average Velocity}= \biggl| \frac{\sum_i \vec{d}_i} {\Delta t_{\text{total}}} \bigg|=\bigg| \frac{\sum_i \vec{v}_i \Delta t_i} {\Delta t_{\text{total}}} \bigg|$$
Where the $\vec{v}_i$ are the different velocities, $\Delta t_i$ are the amounts of time spent at each velocity, $\vec{d}_i$ are the individual displacements ,and $\Delta t_{\text{total}}$ is the total amount of time.
To get the average speed, you take the magnitude of the individual displacement vectors, then sum and average them, giving this formula:
$$\text{Average Speed=} \frac{\sum_i |\vec{d}_i|} {\Delta t_{\text{total}}} = \frac{\sum_i |\vec{v}_i| \Delta t_i} {\Delta t_{\text{total}}} $$
The difference is the order in which you take the absolute value and do the sum. This makes a big difference in some cases: the average velocity of helium atoms in a stationary balloon is $0$, but the average speed may be hundreds of $m/s$ (depending on the temperature).