Can open, unsafe nuclear fusion reaction burn the atmosphere? I happened to hear people saying that the nuclear fusion bomb tests could set the atmosphere on fire. I have some serious doubts about that - but I have no facts.
Nuclear fusion reaction requires $15*10^{6}$ kelvins to start. If we produce such temperature in "open air" would the atmosphere become a fuel for further fusion? Shouldn't the whole thing just be torn apart by its terrible pressure?
 A: Summary: the main reactions in the air involve nitrogen, and in the sea, involve deuterium. Based on the knowledge Bethe had back in the 1940s and making very optimistic assumptions, runaway fusion in the air seemed to be impossible, but with a small safety factor less than 2 if you were to use a fusion bomb with a radius of 3 meters of liquid deuterium. That would be a superbomb vastly more powerful than any bomb we would conceivably make. Runaway fusion in the ocean never was plausible.
However by 1975 in a comment by Dr Gilbert on a paper on the topic, then it was clear that the atmosphere is nowhere near dense enough for a sustained fusion reaction even if the nitrogen reaction had the same energy yield as deuterium tritium fusion (the most reactive known fusion reaction) because of energy losses - and as for the sea, the energy losses for a fusion reaction would make it impossible even in a sea of pure D2O instead of H2O. The energy losses are too great for sustained fusion at the pressures we can attain in an Earth ocean.
This shouldn't be too surprising. After all in the early solar system especially in the first billion years or so the Earth was frequently hit by large impactors of one hundred kilometers in diameter or more. None of our nuclear bombs come close to producing those levels of heating of the atmosphere or ocean, and obviously they didn't cause sustained fusion reactions in the atmosphere or the ocean. After all there is lots of water in the ocean and it hasn't all been converted to helium, and the atmosphere hasn't been converted to magnesium - we'd surely see the signature of such an event even if it was later replenished somehow. Even the Chicxulub impactor was about 100 million megatons, in the energy it released, or two million times more powerful than the Tsar Bomba. See UT Austin scientist reports results from study of Yucatan crater linked to mass extinctions of dinosaurs
It is possible for brown dwarfs to have sporadic deuterium fusion but that is at much higher pressures in the cores of these objects. See THE DEUTERIUM-BURNING MASS LIMIT FOR BROWN DWARFS AND GIANT PLANETS
DETAILS
There's a good account here for the historical background and quotes and it summarizes the reactions they considered:

*

*(The Impossibility of) Lighting Atmospheric Fire
Dongwoo Chung, February 16, 2015, submitted as course work for Stanford university.
There are two competing accounts of how seriously they took it back then, both probably over dramatized in the telling.
Bob Serber:

Edward [Teller] brought up the notorious question of igniting the atmosphere. Bethe went off in his usual way, put in the numbers, and showed that it couldn't happen. It was a question that had to be answered, but it never was anything, it was a question only for a few hours. Oppy made the big mistake of mentioning it on the telephone in a conversation with Arthur Compton. Compton didn't have enough sense to shut up about it. It somehow got into a document that went to Washington. So every once in a while after that, someone happened to notice it, and then back down the ladder came the question, and the thing never was laid to rest.

Bucks interview with Compton

During the next three months scientists in secret conference discussed the dangers of fusion but without agreement. Again Compton took the lead in the final decision. If, after calculation, he said, it were proved that the chances were more than approximately three in one million that the earth would be vaporized by the atomic explosion, he would not proceed with the project. Calculations proved the figures slightly less - and the project continued.

As he says:

Both accounts certainly have an appealing dramatic flair in their respective ways, but when they paint such different pictures of the discussions involved, we must consider their exact details lost to posterity.

DETAILS OF BETHE'S CALCULATION
Dongwoo Chung seems to have made some minor numerical errors in his summaries of the paper, perhaps because the text is hard to read in places. So I'll go to the paper itself for the calculations.
Ignition of the atmosphere with nuclear bombs.
In short the main reactions in the air are
N14 + N14 → Mg24 + α + 17.7 MeV
Bethe calculates a safety factor  of about  1.6 at about 10 MeV
However he works out a mean free path in air of 57 meters, so a region of at least 57 meters in radius needs to be heated for sustained fusion.
To heat up so much atmosphere to 10 MeV he works out needs 1,500 tons of fissile material to be burnt (he doesn't say if this is u235 or plutonium). But typically only 1% goes into heating up the air, so that would require 150,000 tons to be detonated at once to reach the 10 MeV temperature.
For a fusion reaction he calculates that to reach 10 MeV over a 57 meter radius would require 3 meters radius of liquid deuterium to be detonated all at once.
[Dongwoo Chung for some reason says it is 7 meters in radius - the text is a bit unclear in places, maybe he just misread it]
There is an additional reaction
N14 + N14 → O12 + C16 + 10.6 MeV
This requires "only" a 1 to 1.5 meter radius sphere of deuterium but the safety factor increases to 2.67
In the ocean the reactions are:
O16 + H1 → F17 + γ
D2 + D2 → H3 + H1
D2 + D2 → He3 + n
D2 + H1 → He3 + γ
But the safety factors here are far higher
UPDATED RESULTS GIVEN BY DR GILBERT IN 1975
These are comments by Dr Gilbert, Deputy Director of Military Application U. S. Energy R&D Administration Washington,
LLL Comments on the  Ultimate Catastrophe

*

*for the atmosphere, the atmosphere is far too low density for a sustained reaction


Simple calculations show that the atmosphere is of sufficiently low density that even with enormously high assumed cross-sections, burn proceeds much slower than the processes  tending to clamp the matter into a low-temperature equilibrium with its radiation. The available energy per unit volume in air from even complete burnup of the atmospheric nitrogen is only sufficient to produce an equilibrium temperature of less than 1.5 kev, with  over 99% of the energy in radiation.

Also explains earlier

The effects of anomalously large cross-sections for nitrogen burning have never been observed in stars, which have the required constituents, high temperatures, and billions of years of reaction time. The reaction, N14 + N14 -> + Mg24 was considered to be the dangerous by Konopinski, et. al, However, the  strong electrostatic repulsion  of the charged nitrogen  ions requires a relative energy  of approximately 8.6 MeV for them to approach close enough to fuse. ... We know of no way to produce temperatures even 10% of those required.


The cross-sections for the N14 (a,p) and O17 (a,n) reactions in the chain Dr. McNally considers "the most dangerous multiplying chain in air" have also been measured and show no resonance higher than 250 mb, more than an order of magnitude too low to sustain any fusion chain reaction, even if sufficient temperatures  could be reached.
...
Even if nitrogen were many times as reactive as DT, the most reactive known nuclear fuel, the thermonuclear energy generation rate at any plausible temperature would still not suffice to overcome the energy losses due to bremsstrahlung radiation and the inverse Compton effect.


*

*For the sea, propagation fails even in a sea of pure D2O under high pressure.


the sea was modeled in the most simple yet conservative manner by assuming it was two percent D2O at high pressure--more than 100 times the actual deuterium concentration. Initial high temperatures near a 500 Mt massless energy source decreased by a factor ~100 in 2 x 10^-8 seconds. model sea produced an additional 0.006 percent of the source energy before the yield production stopped. The actual deuterium  concentration in sea water would have decreased even this minute burn by a factor of approximately 20,000. In fact, propagation failed (by a large margin) in a model sea of pure D2O under high pressure!

A: From what I have read in "American Prometheus: The Triumph and Tragedy of J. Robert Oppenheimer" Teller was the first one to express this concern before the Trinity test.
Also quoting from: http://www.sciencemusings.com/2005/10/what-didnt-happen.html

Physicist Edward Teller considered another possibility. The huge temperature of a fission explosion -- tens of millions of degrees -- could fuse together nuclei of light elements, such as hydrogen, a process that also releases energy (later, this insight would be the basis for hydrogen bombs). If the temperature of a detonation was high enough, nitrogen atoms in the atmosphere would fuse, releasing energy. Ignition of atmospheric nitrogen might cause hydrogen in the oceans to fuse. The Trinity experiment might inadvertently turn the entire planet into a chain-reaction fusion bomb.
Robert Oppenheimer, chief of the American atomic scientists, took Teller's suggestion seriously. He discussed it with Arthur Compton, another leading physicist. "This would be the ultimate catastrophe," wrote Compton. "Better to accept the slavery of the Nazis than run a chance of drawing the final curtain on mankind!"
Oppenheimer asked Hans Bethe and other physicists to check their calculations of the ignition temperature of nitrogen and the cooling effects expected in the fireball of a nuclear bomb. The new calculations indicated that an atmospheric conflagration was impossible."
Bethe apparently then convincingly showed that the atmosphere would not be set on fire by  a nuclear bomb.

A: I'd simply like to add to physicsphile's answer. 
The primary source for this question is
Konopinski, E. J; C. Marvin; Edward Telle, "Ignition of the Atmosphere with Nuclear Bombs", Los Alamos National Laboratory technical report #LA-602
It shows that the answer to the OP's question is "highly unlikely". It does not prove impossibility. It's an interesting read from the point of view that these were the calculations and reasonings that the whole future of life on Earth was decided with. 
As a physicist, I would say the document is highly sound. Altogether acceptable for making decisions about money expenditure of any kind, even sound enough that one would OK an experiment that could risk even hundreds of lives with (although it is hard to think of a realistic example). But it's a little scary to think that the whole future of life on Earth was decided with it.... 
So let's look at the experimental data. We haven't ignited the atmosphere yet. I think this experimental fact is important for your question: as I understand it, the fine details of the explosion dynamics are to a large degree found by trial and error, and such experimental data are all classified anyway. But the following comments are probably relevant. The biggest bomb to date was the Soviet Tsar Bomba, which let slip $2.4\times10^{17}J$, or $2.6{\rm kg}$ (that's right, kilograms!) of energy (57MT TNT equivalent). The fireball from this monster was eight kilometres across. At this size of bomb, you have probably reached a scale where bigger bombs are going to mean a proportionally bigger volume of space at roughly the same temperatures (on the order of $10^8{\rm K}$). Moreover, Edward Teller calculated that at yields not much higher, the effect of increased yield (as far as the atmosphere is concerned) is negligible: a big chunk of the atmosphere around the blast is accelerated to Earth escape velocity and is lost into space, so adding yield simply means that the escaping gas is going to escape faster: it's not coming back once it reaches $11{\rm km\, s^{-1}}$, so what happens to it is irrelevant. 
A: Around the 60s, a treaty was signed to ban development of nuclear fusion devices with yield greater than about 50 MT (don't remember exact number), in order to prevent fusion of atmospheric hydrogen, thus the uncontrolled multiplication of the device explosive yield. That was before the Threshold Test Ban Treaty was signed in 1974 and entered into force in 1990.
