Is it possible to have mutually incommensurable but equally valid theories of nature which fits all experimental data? The philosopher of science Paul Feyerabend defended this seemingly outrageous thesis and made a very strong case for it. In such a case, is it impossible to decide which of the incommensurable competing theories is the "right" one? In that case, does "anything goes"?
Generally, two such theories are considered equivalent in the logical positivist sense, you choose between them for convenience, and convenience means you use the mathematically simplest form. So for example, you might insist that there is an unobservable ether which dictates the true rest frame, but you just can't determine which frame is the right one even in principle because objects behave with an effective Lorentz invariance.
So you could have relativity, or you could also have an unobservable rest frame, and the two positions seem superficially different. But these positions are positivistically equivalent, so they are equivalent in the deepest sense of the word. What you should say then is that both are true, but relativity is more true, because it is simpler to formulate the symmetry without the extra baggage required to define an unobservable rest-frame. This rest frame is extraneous and arbitrary and would complicate the description of a system which is moving.
The principle that only testably different theories are distinct is called positivism, and philosophers debate positivism endlessly, since if you take positivism seriously, any two philosophical positions which are only different in ways that make no difference to experience are actually identical positions, and one should not consider them different. Any inference you can make in one philosophy you can equally make in another. This means that the metaphysics you choose is like a choice of gauge in electromagnetism--- it frames the definitions of your concepts, giving them meaning relative to your ontology and metaphysics (the computational structures you use to make your statements meaningful), but so long as the predictions are the same, the two ontologies and metaphysics are interchangable. The idea of philosophical gauge invariance moots many of the questions philosophers like to ask, like "Is solipsism true" or "Am I a simulation", as these questions amount to no more sense than the analogous debates of "I like Landau gauge" vs. "I like Dirac gauge" in physics.
To determine which positivistically equivalent formulation one should use, you use Occam's razor--- you formulate the predictions using the mathematical formalism which is simulatable with the shortest computer code. You can't do this entirely, because finding the shortest code is the most unsolvable of undecidable problems, but this is a precise statement of how one chooses between two different formulations of the same theory, where two different formulations are equivalent in the sense of positivism, after you map them to experience, but not in the sense of identical mathematical structures occuring in the formalism.
From your link:
So the decision is based on accuracy, and for physical theories experimental accuracy. As time goes on, accuracy on measurements increases as well as methods of measuring are improved; there will always be propositions for measurements which will distinguish these theories once more accuracy is obtained.
Two disparate in the beginning theories about physics, as time progresses have often been found after long thinking by theorists that they are a subset or a transform of each other; in that case the problem is eliminated.
It is only a philosophical question, imo, not a practical one.
The philosopher Saul Kripke has come up with a solution using modal logic and the possible worlds semantics. For each model or theory which fits all experimental data, assign a possible world. If a proposition P is true in all possible worlds, $\square P$. If a proposition Q is only true in some possible worlds, $\Diamond Q$. Philosophers have analyzed modal logic in incredible detail and come up with many deep insights. Incommensurability means $\Diamond P \wedge \Diamond Q$ but not $\Diamond (P\wedge Q)$. The possible worlds where P is true are not the worlds where Q is true.
Positivists will tell you to only consider those propositions which hold in all possible worlds, $\square P$. The problem is there are too many examples where to prove, $\square P$, you have to make use of $\Diamond Q$ type lemmas as intermediate steps. That kind of undermines the case for positivism.
The answers so far really seem to misunderstand the question, and it's probably because it's flawed.
I really don't know how username anna v comes to conclude "So the decision is based on accuracy,..." from "if theories are incommensurable, there is no way in which one can compare them to each other in order to determine which is more accurate.".
Taking a positivist viewpoint gives an answer to your second questions "In such a case, is it impossible to decide which of the incommensurable competing theories is the "right" one?". But that question is only sensible at a point where you are already in a position to compare.
but if they are incommensurable then how would you make sense of equally valid? So the idea of "agreement over experimental data" is extremely problematic. The concept is all about two theories who can't be compared because the respecive beings who use them don't share a language about the physical world. They talk about different things and the idea is that there is no common ground so one can't judge validity or even understand the results of the other. I'm not going into the question if that situation is possible, but maybe the ideas of Duhem would be a soft point to start and then you could reformulate the question if you come up with one.