I truly apologise if this has been asked to death somewhere, I imagine it has, but I'm yet to find an answer that completely satisfies me.
In short, I don't see why our chosen inertial frames are "correct". They're defined as those that do not require "fictitious" forces to explain motion, but that's also relative, no? Indeed, the forces seem pretty real in the "accelerating" frames, after all. You could say there's no identifiable force causing motion in those frames, but just because we can't identify a cause doesn't quite convince me.
Setting aside the fact that even "stationary" frames on Earth are accelerating and that we already make this sort of approximation, I just don't see how we can claim there to be an absolute 0 acceleration not defined in some relative way. In our case, relative to our (seemingly) arbitrary laws.
All of our inertial frames have the same laws of physics, but so do all frames with any given acceleration relative to our inertial frames, do they not? An acceleration doesn't preserve our laws, but it does change them, and from there, Galilean transformations (or some tweaked equivalent?) should preserve those other laws, yes?
Is there any real reason other than for convenience to assume our definitions of inertia? I feel like any argument I've heard to the contrary can be refuted by simply tweaking some law in some way, or some definition. I suppose it's ideal to assume the simplest laws work ok, but, returning to the example of the approximate frames we use, our understanding of forces and causes is always evolving, and frames that feel or seem inertial are regularly shown to not be thanks to discoveries, like the Earth's acceleration around the Sun, our solar system's "acceleration" through space, and indeed the curvature of spacetime nullifying the notion of gravity as a causing "force". Would it be fair to say we're just defining things as we go along, and in terms of the identifiable causes we can find??
I also apologise if this question isn't very practical - I'm from a mathematical background, after all...