For example, is $F = ma$ really an exact formula, or is it an approximation? I know a lot of formula's come from taking the first few terms of a Taylor expansion, so I was wondering if the simple formulas such as the one stated are exact or approximations.
Physics tries to model the real world by trying to find a model which best fits observations made in the real world. These models will be limited by the accuracy and range of the observations. So it will always be possible that better models will be found when any of those increase.
For example as stated in the answer of Dirk Bruere, relativistic effects occur near the speed of light, so for a lot of practical applications on Earth $F=ma$ will still be a very good approximation. Also general relativity turns out to better predict the motion of planets then Newtonian gravity
Another example of a model which improved after better measurements is that of black body radiation, namely Raleigh-Jeans law was only a good model for low frequencies of light and Wien's law only for higher frequencies.
Planck's law united the two and seems until now still a very accurate model. For example see the spectrum of the cosmic microwave background radiation spectrum.
Well, in that case it is clearly an approximation because it does not take into account Relativistic effects.