Do we ever talk about rate of change of momentum with respect to movement in spacetime? That should be a good quantity, given that spacetime is more fundamental than time.
Do we ever talk about rate of change of momentum with respect to movement in spacetime?
Yes. In spacetime just like time and space are unified into one spacetime so also energy and momentum are unified into one concept called the four-momentum. The four-momentum is a four dimensional vector where the first element is energy and the remaining three elements are the momentum with energy having the same relationship to momentum as time has to space.
Equipped with the concept of the four-momentum we can define the rate of change of the four-momentum. This quantity is called the four-force and has exactly the properties that you would expect in a relativistic generalization of force. The timelike component of the four-force is power. So in relativity the four-force unites power and force in the same way as spacetime unites space and time.
Force was defined a long time before special and general relativity, as models of observations and data came on the scene.
Force is defined in one of the laws of Newtonian mechanics.
In an inertial frame of reference, the vector sum of the forces $F$ on an object is equal to the mass $m$ of that object multiplied by the acceleration $a$ of the object: $F = ma$. (It is assumed here that the mass $m$ is constant.)
In physics theories, laws are extra axioms imposed on the solutions of the differential equations so as to pick those solutions that fit observations and data.
Then why is the definition of force biased towards time?
Because that is what the observations and data need for the theory of mechanics to fit experiments and observations and to predict future behavior. Classical mechanics is validated over and over again in the phase space region of small (with respect to velocity of light) velocities and small masses.
So if you have a path through spacetime, there is a natural length to that path—except you have to choose whether you want a timelike length (in which case the path never goes faster than light) or a spacelike length (in which case it always does.) If you try to mix these you typically get complex numbers involved as you take the square root of negative intervals.
Assuming you like causality for everyone, timelike paths make more sense and their length is known as proper time, measured by taking a clock on the spaceship as it makes that precise journey.
This idea that particles stay within the light cone and therefore have time like paths is why you has more interest in time derivatives rather than space derivatives in relativity. Space derivatives make some sense for fields that are spread out over spacetime, see also (relativistic) classical field theory textbooks for more on this: but for particles, spatial derivatives make no sense.