Before we start, let's assign a "name" to each integral (just for convenience):
$$I_1=\frac{1}{\Delta{x}}\int_{x_1}^{x_2} F(x) \mathrm dx \qquad \text{and} \qquad I_2=\frac{1}{\Delta{t}} \left| \int_{t_1}^{t_2} F(t) \mathrm dt \right|$$
Also, throughout the answer, I will consider the force $F$ to be the net force on the object. I will also be assuming that the mass of the body doesn't change throughout the motion. This way we lose a bit of generality but it becomes way simpler.
Intuition
As you already observed that the average $I_1$ represents the change in kinetic energy divided by the displacement ($\Delta K/\Delta x$). This quantity gives us the answer of the question: What constant force should I apply over the distance $\Delta x$ such that the body gains a kinetic energy $\Delta K$? In other words, it is the average force which would yield the same kinetic energy change (as the original) when applied during the same displacement.
Similarly, the second average, $I_2$, gives us the answer to the question: What constant force should I apply over a time $\Delta t$ so that the body's momenum changes by $\Delta p$ (or equivalently, the body's velocity chnges by $\Delta p/m$)?
Now, we can observe a few differences between both the averages. First, $I_1$ is a displacement average whereas $I_2$ is a time average. Secondly, the change in velocity (or momentum) cannot be independently used to find the change in kinetic energy, which implies both the averges are independent and can independently take different values. Thus there is no reason to believe that both of them are, in any way, equal. However, if the force is a constant, then both the averages will be equal to the constant force. Thus, in uniformly accelerated motion, both the averages will turn out to be equal.
Mathematical Analysis
Let's try to analyse both the averages. It is clear that:
$$I_1=\frac{\Delta K}{\Delta x}=\frac m 2 \frac{(v_{t_2}^2-v_{t_1}^2)}{\Delta x}$$
and
$$I_2=\frac{\Delta p}{\Delta t}=\frac{m(v_{t_2}-v_{t_1})}{\Delta t}$$
Now let's find the ratio of both these averages ($I_1/I_2$):
$$\frac{I_1}{I_2}=\frac{\Delta t}{\Delta x} \frac{(v_{t_2}+v_{t_1})}{2}\tag{1}$$
But $\Delta x/\Delta t$ is the time average of velocity over the time interval $\Delta t$, thus equation $(1)$ can be rewritten as
$$\frac{I_1}{I_2}=\frac{1}{v_{\text{average}}} \frac{(v_{t_2}+v_{t_1})}{2}\tag{2}$$
Thus as you can see, this ratio need not always equal one, which implies that $I_1=I_2$ is not necessarily true. However, in the special case of constant acceleration (where $F(t)$ is a constant, which implies that $a(t)$ is a constant), the time average of velocity comes out to be the average of initial and final velocities, i.e.
$$v_{\text{average}}=\frac{(v_{t_1}+v_{t_2})}{2}$$
Substituting this in equation $(2)$, we get
$$\frac{I_1}{I_2}=1\Longrightarrow I_1=I_2$$
Note: The above analysis, in no way implies that $I_1=I_2$ only when the motion is a uniformly accelerated one. There can be many other scenarios where $v_{\text{average}}=(v_{t_1}+v_{t_2})/2$ even if the acceleration isn't a constant.
In fact, when we divided $I_1$ by $I_2$, we implicitly assumed that $v_{t_1}\neq v_{t_2}$. If $v_{t_1}=v_{t_2}$, we would get a $0/0$ indeterminate form. Although, we can still compare the values of $I_1$ and $I_2$ under this condition, and these simply turn out to be $I_1=I_2=0$. Thus no matter how you change your velocity, if the initial and the final velocities are equal, then both the will become equal to $0$. This is intuitive as well, since in this case, both, the work done and the momentum change are $0$, thus yielding $I_1=I_2=0$.
Summary
Thus, both the averages, in general, aren't equal, but in the case of uniformly accelerated motion, both the averages turn out to be equal.