I think anyone who says "there's no need to do experiments, we can simulate everything!" either:
- Doesn't know what they're talking about
- Is trying to sell snake oil
- Is a scientific fraud trying to push pseudoscience as actual science
I have never seen a serious, honest scientist claim that simulation is sufficient substitute for empirical evidence, even for very simple experiments.
Washington Post news article
Looks like an excellent example of #1.
There is a reason that a critical component of the scientific method is experimental verification of hypotheses. To claim that simulation is as good as an experiment is ridiculous: What if the simulation is wrong? What if there's bugs in the code? What if the assumptions you based it on are false? What if the approximations inherent in your theory (all scientific theory by necessity involves some approximation of reality) cause catastrophic divergence from real-world result? What if there are hidden variables nobody was aware of?
And most importantly, how on Earth will you know the results are real unless you've went and tried it? Even with a competent simulator there are issues, and academics produce notoriously poor-quality code. Without experimental validation, you might as well be making it up. From the article:
But a former nuclear weapons designer, who spoke on the condition of anonymity because he is still in the government, offered a more cautious view. “To say the calculations are better than underground testing is silly,” he said. “If you want to know if something works, you have to test it. The calculations are good, but the issue is one of risk. How good do you think the calculations are?”
And:
Sen. Jon Kyl (R-Ariz.), who has long opposed the treaty, said: “Computer simulation is a part of the stockpile stewardship program, which scientists say has been helpful. One told me it produced good news and bad news. The good news is that it tells us a lot more about these weapons than we ever knew before. The bad news is that it tells us the weapons have bigger problems that we realized. While computers are helpful, they’re not a substitute for testing. That’s why, even though we’re not testing right now, we should not give up the legal right to test.”
And:
Jeffrey G. Lewis, a nuclear weapons expert and the director of the East Asia Nonproliferation Program at the Monterey Institute of International Studies, said that years of underground nuclear tests helped show weapons designers that the bombs worked under certain conditions, “but they could never fully explain how or why.”
Emphasis mine - although Jeffrey Lewis is apparently in favor of the policy, his remark illustrates the crucial point: The universe operates on complex laws. Not all of these laws are known. It is impossible to guarantee that a simulation based on known laws will agree to reality. In some cases the disagreement may be insignificant, but there is no way to know a priori that it will be for any given case.
EDIT to add yet one more problem with relying on simulations: If you look at the past experiments conducted to validate simulations, you may see that they conform very closely and get the idea that the possible sources of error I mention are insignificant, and that I am making a mountain out of a molehill. However, you would be ignoring certain important nuances.
The moment you start eschewing empirical validation, the character of the research work: It is a lot easier to make predictions when you know nobody will actually test those predictions and catch you if you're wrong. Moreover, you fall into a bizarre trap similar to skipping controls: You have some simulations that you validate, and sometimes they turn out false and you have to scrap your manuscript, but you have other simulations that you don't validate, and they never need to be re-done. What is the point of the validation then, if all it does is slow you down? (To connect with the analogy - "what is the point of doing controls, if all they do is make experiments work less often?")
Of course, the answer is evident: The experiment is worthless without the controls (this holds even if over your scientific career you have performed 1000 controlled experiments, and never once have the controls given you any new information). Likewise, the simulation is worthless without the empirical validation. But if one accepts that empirical validation is dispensable, then a contradiction arises: Why not dispense with validation even more? After all, you only win and you "can't lose". Because of this, and coupled with various human flaws to which all scientists are subject, once empirical validation stops being "always mandatory, no excuses", it won't be long before everybody is coming up with made up theories that have nothing to do with reality and no trace of genuine science remains in the community. This is the big danger that "really" makes it important to do empirical validation of simulations.
This isn't to be interpreted as "we have to do nuclear testing or all science will be ruined forever". I don't personally mind that the testing stops. However, once the decision to stop testing stops, a serious person should admit to themselves that certain scientific knowledge which was to be obtained from those tests is now inaccessible, and can no longer be obtained. Saying "we're stopping the tests... but it's okay, we can do simulations!" is like asking to have your cake and eat it too. It's not happening, and you are fooling yourself if you think it is.
And I'm also not trying to claim that making simulations is a pointless business. It's very useful work which can produce valuable scientific results. But it is important to remember that results from a simulation apply only to the simulation, not reality, because reality is not the simulation, does not necessarily resemble the simulation, and most likely differs from the simulation in unpredictable ways. One cannot take the results of a simulation, and claim they apply to the real world: One can only make the argument for performing a proposed real-world experiment, based on the results of the simulation.
But if the experiments are not an option, then the simulation can no longer be connected to reality. At that point the researcher is stuck investigating only the phenomena of his simulated world, not the real one. Once again, there is nothing wrong with that, and arguably the whole discipline of mathematics is concerned with investigating an imaginary world, yet it has produced many findings useful in the real world. However, with the word "science", one is wont to associate investigation of the real world, rather than an imagined, simulated world.
Also, as you can easily see from reading the article, the decision to stop testing is politically motivated. Saying that you decided to stop testing is expedient in the current political climate (even though it's unscientific). Removing the strict test of empirical validation is expedient because it allows easier manipulation of results. Neither On the other hand, a stalwart devotion to scientific principles is unlikely to be appreciated by much of the electorate, and since nuclear war is unlikely during the current administration, losing the ability to design effective nuclear weapons (and likewise for nuclear power plants) is not seen as a big deal. Therefore, the administration makes a [politically] correct decision to stop testing, and claim that simulations are good enough anyway.
