Well, you generally have a few fundamental equations, and you try to build a theory upon them. For example, one can consider the gravity equation $F=\frac{Gm_1m_2}{r^2}$ to be a fundamental. How was it derived? It wasn't -- it was experimentally determined to a known accuracy. Newton's laws can be considered to be "axioms" as well, though the Lagrangian formulism of Physics is more pleasing to look at axiomatically (some very simple statements involving energy are taken as axioms, and the rest is pretty much mathematically derived).
More modern theories like the Standard Model assume entities behave according to some mathematical relation -- again, this is your axiom (I'm not too familiar with SM, so I'm not too clear about this).
In the end, the only way to check is via experiments. It's technically the same with mathematics-- you "test" your axioms against your logical facilities. The axioms are pretty simple and sometimes silly at first glance, so you never know that you're doing this. On the other hand, in Physics, the axioms are more complex and don't sound as "silly". The experiments required to verify/come up with these are correspondingly more complex, and they don't give a completely definite result -- we can only say that "equation X is valid to Y accuracy" or something like that.
If we call the fundamentals "axioms", then yes, Physics is rigorous. But these axioms are on pretty shaky ground themselves. For the past century we've been trying to tweak these axioms/postulates so that we get a system consistent with the real world.
Another way to look at it is that we just port over all axioms from Mathematics, postulate a few things (what a force is, what momentum is, etc), and everything else is a conjecture that's been verified to a certain degree. Like Goldbach's conjecture.
I think that the second interpretation is the one commonly accepted -- I've never seen anything labelled as "axioms of physics", though certain things like special relativity can be derived in an axiomatic fashion (start with space and time, then go on)