Set theories used in mathematical physical theories For most theorems, this is unimportant - a typical problem will have the same solutions in NFU, ZFC or some other theory, since said theories got popular precisely because they give pretty much the same results that were derived in "naive", earlier mathematics, and so fit our intuitive understanding of what mathematics should look like. 
However, these theories aren't identical, and there are several statements whose truth value differs depending on which set theory you choose. Most of such statements aren't of any importance to physics - they are purely mathematical exercises designed to create problems to set theories discussed (or aren't important for the purposes said theory is created for and so there is no point in having them agree between the theories).
In pure mathematics this variety doesn't matter, since from pure mathematical viewpoint, there is no "correct" axiom system.
This changes when we switch to mathematical physics, because from mathematical physical viewpoint axiom systems can be incorrect (when they don't agree with physical reality), and theories are abstractions of the real world.
Each set theory defines how objects known as "sets" behave. The behaviour is different in different theories, and sets of ZFC are different than sets of NFU - although both of them are based on objects well know from our intuition, they are different since they take different paths of abstracting these intuitive objects. A mathematical theory in physics (like the Lagrangian formulation of classical mechanics) should use only one of these descriptions of behaviour, because using different definitions for different problems makes said definitions meaningless (which definition is used in the axioms? Are sets which define the concept of an integral in the principle of least action sets of the ZFC or the sets of NFU?) - if we want to have a meaningful concept of a set, we can't switch its meaning while using a single formalism. 
Now, here is the problem: when I read about, as an example, like above, Lagrangian formalism, no book talks about which set theory it uses. Most use the concept of a set without defining its behaviour, which creates the philosophical problem described above (i.e. which "sets" are we talking about when we are talking about "sets"). This isn't unimportant, because a lot of objects, like integrals or vectors, in the end need the concept of a set to be understood (the definitions of functions and of vector spaces in these cases). So we end up with loads of different formalisms, the one based on NFU, the one based on ZFC, and so on. 
This wouldn't matter if the results were the same in each of them and every problem asked in Lagrangian formalism would be solved with the same result, no matter the set theory - after all, the purpose of a model is to solve the problems in order to describe and predict the behaviour of physical objects in the real world. If the results were the same in each model, the choice of a set theory would be pointless philosophy without any meaning in physics.
The thing is, they aren't. 
Principle of least action could be described in set theories which in the end lead to different results of problems, which is at least unnerving considering that this principle tries to describe objective truth.  I don't have any such problem at hand - somebody more experienced in creating exercises, could, probably quite easily, think of one considering that there are obvious differences between the theories, but I don't need to think of any, since it's irrelevant to my discussion - we can't prove there isn't a single problem concerning the principle of least action which has different results in different popular set theories (how could we?), so the problem of different results still stands.
Summary of my thoughts is: depending on which set theory one uses, a mathematical physical theory can give different results to problems posed. At the same time, it tries to be an abstraction of reality, not just a disattached mathematical theory. 
How is this problem (of not specifying the definitions of objects which have very different definitions used in mathematics, leading to different implications of physical theories) solved in mathematical physics?
 A: Tl;dr: Physicists would care about which set theories to choose iff there is an observable difference between the choices. That's less likely than you think, because "observable outcomes" are always finite numbers, but nevertheless, somebody should look into that (indeed they do).
Almost all mathematical physicists would agree that this is the only reason to care. For the sake of rigour, they implicitly choose the most common set theory ZF(C) due to time constraints.

Let's first give an answer to the following question:

Do we need to care about the foundation of set theory in physics?

Detour to the scientific method
Science is in many ways a social and personal endeavour (see my detour below). This is crucial, because it influences the way people think, but in this section I want to quickly sketch a few more "objective" ideas - in that they are more or less universally accepted ideas: The scientific method. 
The "objective" point of science accoring to that method is predicting outcomes. Those can be past outcomes, but it should also work for future outcomes. This is why science is "useful". Because of the complexity of life, this has to be broken down to small units - experiments. The nature of experiments is all but simple and it is certainly more than "observation", because our senses are not very good (think about optical illusions). In order to remedy this, we invent measures and measuring machines so that, in the end, the outcomes of experiment is usually a bunch of numbers on a computer screen. That means all outputs are always computable. 
(Mathematical) Physicists
Most physicist view mathematics as a tool. They marvel at the effectiveness in physics, but all they want to do is to approximate physical theories. In fact, if you don't work at Theories of Everything, you are well aware that you only work with effective theories. An effective theory is by construction only an approximation. Many mathematical physicists work on effective theories themselves and they agree. 
At that point, I would argue, the question of whether we need to consider the foundations of set theories depends on the world view. Here are the most common alternatives:
1st idea: Since everything we ever measure is computable numbers and the number of excitations of quantum fields is likewise finite and computable (due to the fact that the universe is deemed finite and everything we think we know about it), a "true" theory of everything would actually only use computable numbers. 
If we agree on natural numbers (and even the set theories you cite normally do), then this means the question of foundations is irrelevant. But obviously, we use functional analysis and the like. Why that? Well, because approximating a finite but huge system by an infinite system is much more convenient. Things become much simpler and we avoid a lot of technical hassle. But at this point, it doesn't really matter which set theory we choose. The Banach-Tarski paradox in ZFC? It's just an artifact of our approximation and has no bearing on the real world. Why should we therefore only use ZF or another system? 
2nd idea: A true description of the world needs the continuum, however, anything we can ever accomplish is an approximation to that true description or even many different approximations that for some reason cannot be welded together. The foundation of set theory may matter here - but to a very limited degree: It matters if and only if two theories would lead to observably different outcomes. Since, once again, all experimental data we ever get consists in computable numbers, this observable outcome must be in computable numbers. I guess most mathematical physicists would see this as unlikely and thus focus on other matters at hand.
3rd idea: Some people believe that the universe is a mathematical structure in need of the continuum and this structure can be found. At that point you need to care about set theory. But once again, you need some way to discern the "correct" set theory foundation by experiments. This should be possible if you believe that the strucute can be found. 
Upshot
In any case, a mathematical physicist will only care for the foundation of set theory, if there are observable differences between two choices. Since all the data we can ever have consists in computable numbers, this seems like a long shot and thus most people don't really care. Some do - you might like the article "Set theory and physics" (paywalled, but there are versions without paywall that I don't link because they go directly to the pdf) and other papers by Svozil. Note that he tries to find observable quantities that differ depending on the outcome.

Does that answer your question? I'm inclined to believe that no, since everything I said so far does not only hold true for mathematical physicists, but for every physicist. The refined question is probably:

Why don't mathematical physicists care about the foundation of set theory to be rigorous?

If you only want to be rigorous to be sure to have a consistent system, you could pick any consistent formulation of set theory. Of course, consistency is nice, because we like to believe that our world is consistent, hence inconsistent theories would be wrong. Hence to achieve that goal, you can choose whatever theory you want - and it's most convenient to do what everybody else does, namely ZFC.
However, that's not the only reason why mathematical physicists believe that rigour is necessary. It might also help to better understand the theory. Examples are given in the answers to this question: The Role of Rigor
Once again, for all of that, it's enough to use the most convenient formulation of set theory as long as you choose any. 

The last objection I can think of is:

But mathematical physicists rarely state which set theory they work with. Shouldn't they do that?

My answer to that would be that 99% of all mathematicians don't do this, so why should mathematical physicists? Why don't mathematicians do it? Let me give a short final detour:
Rigour in mathematics:
Pure mathematics is rarely rigorous. Reading an advanced paper in, say, Functional Analysis, you'll have to fill in a lot of gaps and the foundations are rarely specified. The closer you come to the foundations of mathematics, the more rigorous the papers become (by nature of the endeavour), but even then, many parts of the proofs are regularly left out. 
Mathematics as personal endeavour
The problem is that mathematics is a human and therefore also a personal endevour after all and for most people, it's a question of gaining knowledge. However, every person - unless forced otherwise - has some level at which he/she considers things "obvious". If you want to "know" anything, you'll have to make assumptions at some point, otherwise you go down the rabbit hole and end up nowhere. Decartes famously ended up at "I think, therefore I am", but it's not even clear that this deduction is valid. 
So you have to start somewhere and you could start at set theory. However, most people will not naturally start there, but consider things like "natural numbers" obvious (you could call this "naive set theory"). They won't really care about whether this is actually consistent or not until they hit a contradiction. Other mathematicians are happy to choose just any formulation and work at higher levels, mostly the one they were taught at. As long as they gain "intuition" about what's going on, they see it as gaining "knowledge" about the field of research. This is also why they leave out certain arguments in proofs - just thinking that they know how it would be done is enough to them (and they are wrong quite often!). This is as I understand it one of the main points of Bill Thurston's famous essay "On proof and progress in mathematics". There is also the question of time constraints: Not everybody can start at the foundations, otherwise no one can build the higher theory.
Incidentally, the personal nature of mathematics is also why many mathematicians feel uncomfortable about proofs done by computers. Using computers to check proofs is fine, but to find proofs? What does the mathematician learn?
Mathematics as social endeavour
Second, mathematics, like everything else, is a social endevour. Both mathematicians and physicists often like to deny this, but research goals, the way research is done and the like are heavily influence by social norms, I will likely do research similar to how my teachers do research. Of course, differences in style will develop, but they will usually be incremental. The biggest changes in how research is done is usually done by a very small minority of extremely idiosyncratic thinkers like Hardy.  
Otherwise, most changes in the level of rigour are due to the fact that something affects the outcome of research: The nineteenth century saw a more rigorous analysis of convergence, since for instance it is easy to construct a class of functions $f_n$ which converges pointwise to another function $f$, yet the arclength of the functions $f_n$ does not converge. Since intuitively we think this shouldn't be the case something is "off". Similarly, paradoxes have let to axiomatic set theories in the 20th century. 
I am sometimes dismayed by this unfortunate lack of foundation, which is why I endorse algorithmic proof checkers and the like. 
A: My answer to this question is heterodox and polemic. I consider the establishment of uncountable infinities in Cauchy sequences and Dedekind cuts as unnatural, unreal, and unnecessary.
Cantor's highly celebrated result, if reinterpreted in the aftermath of Cohen's Theorem and Goedel's work, is a extremely obvious yet important statement: incomputability allows us to state that certain results are not even well formed. Cantor's argument, about the missing incomputable real numbers in comparison to the rationals, shows that there is a huge gap between computable and incomputable objects. Computable objects are the only thing we can really study and analyze. No computable (i.e. constructible) result in any branch of mathematics requires the use of the incomputable.
Simply stated, no result, for example in calculus, that is about computable objects (like a single real number that we can approximate as much as we reasonably need) needs Cauchy's blunder (which remains in understanding calculus until today). A single Cauchy sequence is fine or any finite/countable collection, but if you really need more you have to ask yourself what it even means since they are by definition incomputable.
Set theory is simple and obvious for finite or countable objects while set theory for the continuum is paradoxical and unintelligent. Cantor's "mistake" was that he wanted to distinguish between the reals and the rationals without rejecting the value of incomputable numbers that dominate the real line. Instead, mathematical techniques like compactness, continuity, etc. are mostly used to provide a route to renormalize functions defined over the continuum such that they become so well behaved that the continuum is irrelevant in their actual analysis. Along the way, we reject well defined objects like the dirac delta and the heaviside step function because they are not simple enough (more generally, functions with finite or countable jumps, but please explain what a function with uncountable infinite jumps is even suppose to mean).
I don't think finite set theory, which is so trivial, has any problems in mathematical physics, so the only question is whether you'd like to spend your life analyzing objects that are by definition outside of your ability to interpret them.
EDIT:
In "computable analysis", the concern is about "computable numbers", for which any desired level of precision of the number is computable. The number itself though can be an uncomputable object (no infinite precision form exists). As such, these numbers themselves are not necessarily computable, just their approximations to a desired level of bit precision. Rationals are computable, reals in general are not, and "computable numbers" may only be computable up to a particular precision, in which case their finite precision forms are of course rational. The reason why terminology exists like this in the field, to my understanding, is because we already have a term for numbers we can compute (rationals), a term must be found to distinguish between the case where a real number is not even computable up to a desired precision, and "computable up to any desired level of precision" becomes a mouthful to use in practical writing.
