To answer this question properly requires probing deep - an exercise in deconstructing space and time. This is, in fact, what mathematicians have done over the past century or more, separating out the different elements of the infrastructure into different layers. From the vantage point of a programmer, this frame of mind is quite familiar: one starts with a base type, and builds from it, derived types, each one adding more infrastructure onto the type it is derived from.
At the rock bottom layer is a set of points. That's Layer 0.
They are endowed with a "topology" which gives them sufficient structure to determine such things as "continuity", "contiguity", "connectedness", "interiors", "boundaries", and so forth. Geometry, pursued at this level, is sometimes called "rubber sheet geometry", because at this low level there is no concept of congruence, shape or similarity. The topological structure is Layer 1. In this layer, there is also enough to conceive of the notion of continuous sequences of points, called "paths" or "curves" or "trajectories".
The geometry should be sufficiently structured to support the notion of "rate of change", so we can talk about gradients, velocities and so forth. The additional structure required is called the structure of a "differential manifold". It makes it so that the geometry can be displayed in a series of maps (an "atlas") that consistently mesh together. So, in the vicinity of a point, it looks and acts much like the continuum of an ordinary geometry on which you can do calculus.
That's Layer 2. In this geometry, at Layer 2, there is now not only enough infrastructure to conceive of a notion of paths, but also such notions as its "gradient" or "speed".
Classically, a geometry is purely spatial and involves only spatial relations. However, since the time (at least) of Galileo, who entertained the idea of a symmetry transform that mixed the time coordinate in with the spatial coordinates (that transform now being called a "boost"; i.e. a transform from one frame to another that is moving uniformly with respect to it), the two have been intertwined. This marriage of the two was essentially an elopement that was not fully consummated until around 300 years later, when it also emerged that a "boost" also mixes spatial coordinates in with the time coordinate and alters temporal relations such that what is deemed to be simultaneous, when boosting to another frame, is no longer simultaneous.
The point of the last paragraph is that the notion of geometry necessarily expands to include time, and so would be rightly called a "chrono-geometry"; the object of its study no longer being a "space" but a "space-time".
In a space-time, one not only has the "paths" of purely spatial geometry, but also sequences of points that ascend in time, which are called "trajectories" or "worldlines". The "gradient" of a path becomes the "velocity" of a worldline. So, now we have enough to talk about motion and to do calculus with it. It is at Layer 2, that the basic law of kinematics emerges: Velocity = rate of change of Position with respect to time.
However, at Layer 2, there is not enough infrastructure speak of the "curvature" of a path or the "acceleration" of a worldline! That requires additional infrastructure - and (voilá!) that's the very infrastructure your question pertains to!
It's called a "connection". A connection does two things. For purely spatial geometries, it endows paths with a notion of straightness by determining whether a direction remains "the same" at different points along the path. A path that keeps the same direction is one that is then deemed to be "straight" and is called a "geodesic" (technically, it is only called an "autoparallel", the term "geodesic" only pertains to Layer 4 below). For the curved surface of the Earth, for instance, the geodesics at two nearby points on the equator heading north would be initially parallel but would both proceed along their respective lines of longitude to the north pole, where they converge and meet. The example of such curves on the Earth is the origin of the term "geodesic".
The extra structure of a connection gives you Layer 3.
In the space surrounding the Earth, the spatial geodesics are well-approximated by the paths taken by rays of light, so that a light beam traces out a spatial geodesic.
For trajectories, the connection determines which motions keep the "same velocity" from one time to the next; i.e. which trajectories are non-accelerating or "inertial". Out of all the worldlines in the chronogeometry, the connection determines a subset of them to be inertial. These, too, are called geodesics.
The chief property of the connection is that through each point in each direction runs a unique geodesic.
The question you're asking amounts to the question: "where does the connection come from" or "what determines the connection"? That is, out of all the possible worldlines or paths traversing the space-time, why are some of them straight or inertial and others not?
In Physics, the extra structure is simply assumed to be an extra layer of infrastructure that's just there. In most physical theories, a chrono-geometry with layers 0, 1, 2 and 3 is generally assumed to be there as a precondition to whatever theory is posed.
For dynamic and geometric theories of gravity, the connection is itself subject to dynamical laws that determine how it changes from one point in time to the next. But that it actually be there, in the first place, is assumed at the outset, not explained away. The outstanding example of this is the Einstein Equations. The Einstein equations are formulated on top of a geometry endowed with Layer 3 infrastructure, so the existence of Layer 3 is there as a precondition. The theory assumes that there is a connection, the equations help determine what it is. But the fact that there is that extra infrastructure of a connection is there, is an assumption or precondition. Left unanswered is why there should be any Layer 3 infrastructure at all.
So, when you're asking "what determines which motions are inertial", whether you realize it or not, you're really also asking about where all these other parts of what comprise Layer 3 come from. That includes the question: what determines which curves are "straight"?
Finally, at Layer 4, one has the "metric". For spatial geometry, this provides the additional infrastructure of angles, congruence, path length (and area and volume), orthogonality and a semblance of the Pythagorean relation. For chrono-geometries it provides the infrastructure needed for the concept of duration, time measure, and of a "space-time orthogonality" relation.
(A spatial direction in a chrono-geometry is orthogonal to a temporal direction if a path oriented in that direction is seen as being "simultaneous" from the vantage point of someone on a trajectory oriented in the time-like direction. So space-time orthogonality gives us a local version of "simultaneity".)
To reach Layer 4 from Layer 2, it is enough to just add in a metric. A metric determines a connection and notions of geodesic and inertiality by the "least distance" and "greatest time" principles.
For paths in a purely spatial geometry with a metric, the geodesics are the paths that provide the shortest connection between its nearby points. I say "nearby" with the example of the Earth in mind. The Greenwich mean line is roughly a geodesic, that wraps around the other side of the world as the 180 degree line. Any two points on it can be traversed between in two ways: one directly, and the other by going the opposite way around the world. Only the direct way is the "shortest" way.
For chrono-geometries, the corresponding notion is that of the "inertial" worldline. These are the worldlines that connect two points on its path in the "greatest" amount of time. Thus, for instance, an inertial motion between the Earth and the moon would register more clock time than a motion that quickly accelerates to a high speed upon leaving the Earth and quickly decelerates back to a stop, upon reaching the moon. The amount of time dilation for the worldline is directly related to how much from inertial the worldline deviated.
When the structure of a metric is added directly onto Layer 2 to reach Layer 4, skipping the intermediate stage at Layer 3, the connection derived from the metric is called a "Levi-Civita Connection" and gives you the required infrastructure for Layer 3. Such a geometry is called a Riemannian manifold - if it is a purely spatial geometry. If it is a chrono-geometry, it's called Lorentzian, which is a subclass of what are called the "pseudo-Riemannian" manifolds.
Pseudo-Riemannian manifolds are a larger class of chrono-geometries that permit two or more time-like dimensions, while Lorentzian manifolds have only one, and Riemannian manifolds have none, but only spatial dimensions.
The space-like or time-like nature of a dimension is determined by the metric itself. The metric gives you the approximate semblance of a Cartesian coordinate grid surrounding each point (the prime example, of course, being a segment of the Earth's surface when it is mapped on a flat sheet with a coordinate grid) ... but with the proviso that the Pythagorean relation goes like $α(Δx² + Δy² + Δz²) - β(Δt²)$ for suitable coefficients $α$ and $β$ (e.g. $α = 1$, $β = c²$). The time-like dimensions carry signs that are opposite from the space-like dimensions in the Pythagorean relation.
The distinction between space-like and time-like directions is metrical: the concept exists only at Layer 4. There isn't enough infrastructure at Layer 3 to tell space-like apart from time-like.
A good down-to-Earth example of this, by the way (literally), is this: what happens if you were to take the flight distances between 4 cities (like New York, Chicago, Miami and Houston) and treat the flight paths as straight lines? That is, what if you were to pretend to be a Flat Earther and pretend that all flight paths were not just geodesics but outright straight lines? Could you fit the distances onto a tetrahedron in a Euclidean geometry? The answer turns out to be no! If you actually look up the distances and run through the calculations, you'll find that they require a 2+1 dimensional geometry to fit into.
(If you expand the exercise to include two more cities, like Los Angeles and Seattle, and treat the 15 flight paths all as straight lines, you may very well find that the 15 distances require a 3+2 dimensional geometry to fit into and that they won't fit in either a 5+0 dimensional space, nor even a 4+1 dimensional space!)
So, another answer to your question is to skip Layer 3 and go straight to Layer 4. The question now becomes: "why is there any metric at all?"
It is possible to introduce both the metric and connection independently onto Layer 2. Then one can distinguish between the "native" connection and the Levi-Civita connection given to you by the metric. The difference between the two connections is then called the "contorsion". If we require that metrical relations be preserved along the geodesics (and inertial worldlines) then the connection is called "metrical" and the geometry, itself, is called Riemann-Cartan.
The distinction between "autoparallel" and "geodesic" is here. If the connection is Levi-Civita, then the autoparallel curves are the same as the geodesics. Otherwise, if both a connection and metric are present - as independent objects - then geodesics and autoparallel curves generally do not coincide. It will be the autoparallel curves that are "inertial" and "straight", while the geodesics will be the shortest or longest-duration curves, but not generally inertial or straight. "Autoparallel" is the more general term, since it applies to both Level 3 and Level 4, while "geodesic" only has meaning at Level 4.
There is also an intermediate layer, Layer 3½, which includes only the conformal part of the metric. This is the metric up to change in scale (and sign). This is enough infrastructure to see the distinction between spatial and temporal dimensions, which Layer 3 can't see, enough to recognize geometric similarity and angles (but not congruence or length), and also enough to distinguish which directions are space-like, past-pointing or future-pointing, but not enough to allow you to define a unique Levi-Civita connection, nor an unambiguous notion of "geodesic", but only "auto-parallel".
In addition, it is also possible to only go part way with Layer 4, by allowing the metric to be of rank less than 4. The geometry for non-relativistic theory is a Newton-Cartan chrono-geometry, its metric being only rank 1: $Δt^2$ ... which corresponds to $α = 0$, $β = 1$. There is no notion of spatial geometry, per se, native to this structure. Instead, one has to resort to the inverse metric, which is given by the structure
$$\frac{1}{α} \left(\left(\frac{∂}{∂x}\right)^2 + \left(\frac{∂}{∂y}\right)^2 + \left(\frac{∂}{∂z}\right)^2\right) - \frac{1}{β} \left(\frac{∂}{∂t}\right)^2$$
rescale it (by multiplying by $αβ$) to
$$β \left(\left(\frac{∂}{∂x}\right)^2 + \left(\frac{∂}{∂y}\right)^2 + \left(\frac{∂}{∂z}\right)^2\right) - α \left(\frac{∂}{∂t}\right)^2$$
and then apply $(α,β) = (0,1)$ to get the Poisson Operator:
$$\left(\frac{∂}{∂x}\right)^2 + \left(\frac{∂}{∂y}\right)^2 + \left(\frac{∂}{∂z}\right)^2.$$
The metric and its inverse have to be treated as independent, though related, objects. The Newton-Cartan space is the chrono-geometry which has these two structures added onto it. This is a generalization of the Riemann-Cartan space-times of Layer 4. Unlike the Levi-Civita connection, a connection in a Newton-Cartan space-time is not uniquely determined by the metric.
This can be ameliorated by embedding the geometry into a 5D geometry equipped with an extra 1D invariant
$$Δx² + Δy² + Δz² + 2β Δt Δu + αβ Δu² = 0$$
with the Proper Time invariant given by:
$$Δs ≡ Δt + α Δu.$$
For each value of $(α,β)$ the 4D chronogeometry embeds into the 5D where the metrical relation/constraint, upon substituting for $Δu$, would reduce to the relation
$$β Δs² = β Δt² - α (Δx² + Δy² + Δz²)$$
suitable for use when $β ≠ 0$. So, $s$ plays the role of historical time and is invariant. For $α = 0$ and $β ≠ 0$, it gives you a 5D geometry suitable for Newtonian physics - called the Bargmann Geometry.
The curved version of this metric is used in 5D cosmology, for $αβ ≠ 0$ (which means both Euclidean 4D and Lorentzian 3+1D), because it can be equivalently described as the metric obtained by substituting the proper time $s$ for $u$ as:
$$Δx² + Δy² + Δz² + \frac{β}{α} (Δs² - Δt²) = Re \left(Δx² + Δy² + Δz² - \frac{β}{α} \left(Δ(t + is)\right)^2\right)$$
a metric with complex time $t + is$. It changes between locally Euclidean and locally Minkowski signature when the sign of $αβ$ changes.
Why am I personally interested, also, in the question of the origin of "inertia"? Well, you've seen the recent reports and videos released from the Pentagon about those strange vehicles that zip about with extremely fast and sharp stop and go actions, leaving behind no sound, turbulence or wake, making a complete mockery of the law of inertia, almost as if they're trying to showboat and flaunt it. Whatever is driving those vehicles, it is as if they found a way to actually shield the effect of inertia - not just in the vehicle itself, but in the surrounding space.
(The reports go much further with this, indicating that the craft were able to start at tens of miles up in the air, zip suddenly down to a few feet over the ocean and come to a dead stop - all in a fraction of a second - without any sound, sonic boom, wake or friction burn.)
Never mind whether it is for real or not. Just the mere idea sparks curiosity and raises the question: whether and how it is possible to accomplish that within the known geometric framework just laid out. Within that framework, it's easy to describe: the vehicles are messing around with the infrastructure provided at Layer 3, altering the connection in such a way as to make the fast stop-and-go motion the one that is inertial at each point along its path, instead of the stationary or slow-moving motions ordinary objects would take in that same setting.
Do we need to go outside known physics for this, and are we seeing a demonstration of that as-yet-unknown physics being flaunted before us?
So, the question you're asking now gets put at the top of the heap: is there something more that determines the connection than just the Einstein law of gravity? Something that can actually be engineered in the way that allows one to shield from the effects of inertia at rapid acceleration and deceleration both within the vehicle and in the area surrounding it?