Why is it often said that any mathematical theory is just an approximate theory of the universe? Wouldn't there be accurate mathematical structures repressing the physical entities of the universe precisely?
There are three reasons why mathematics is stated as an incomplete description of physics. I list them in order from pragmatically physical to more philosophical.
- Any calculation, any actual prediction of physics is based on a mathematical description that is known to be a mere approximation. You could conjecture that you have the complete list of mathematical objects relevant for any physical situation (which the overwhelming majority of physicists believe you don't). But you never use them in their full potential, because that would ultimately mean describing all physics of the whole universe, from the tiniest speck of dust to the largest supercluster.
- Mathematics itself cannot ever self-explain what does it describe. How is the symbol $v(t)$ connected to reality? I don't see any symbol "t" or "v" floating in space to tell me how to use it. In the end, language and it's common sense provide a bridge between mathematics and actual physics. The link between mathematics and physics is provided by labels created by people. But there is an inherent approximateness and fuzziness to things created by people (however small and negligible in many physical examples).
- Any amount of labels isn't the object or reality itself. If it were, you could throw an object away after knowing all the labels. It would be enough to describe a color by the photon wavelength to a blind person to provide the full experience (see qualia). But no mathematical object has anything to do with any real object. The real line is just a set of sets which contain empty sets. A function is just a map from the real line to the real line. Modern mathematics is just empty delimiters and mappings, and the higher order structure we usually work with can be explicitly connected to these through a "verbal" ("logical") sequences of symbols assembled according to certain rules. Thus, mathematics provides only models of reality for much more fundamental reasons than mentioned in the first point.
Whenever you come up with a theory (eg: Newtonian mechanics), it has some physical domain of validity, and then you come up with the next (better) theory (eg: relativity), and so on. This process might not have a "fixed point".
At least if you had a fixed number of things to explain, then you might be able to consider iteratively simplifying it to an extent where you have few assumptions and predict everything else. But the universe is a large place (not just in space as one thinks naively, but also in scales -- one has interesting physics at all length scales from the extremely small to the extremely large), and there are potentially (infinitely?) many things we're not aware of that we'll have to keep incorporating and learning and improving our models.
To my understanding, there is no a priori reason for this process to have a fixed-point i.e. an end to which you converge. And even if such an end exists, there is no saying whether you will get there in finite time -- you might take infinite time to reach that absolute theory... so "forever" you will be short of that and your models will be approximate.