Rigorous arguments are very similar to computer programming--- you need to write a proof which can (in principle) ultimately be carried out in a formal system. This is not easy, and requires defining many data-structures (definitions), and writing many subroutines (lemmas), which you use again and again. Then you prove many results along the way, only some of which are of general usefulness.
This activity is extremely illuminating, but it is time consuming, and tedious, and requires a great deal of time and care. Rigorous arguments also introduce a lot of pedantic distinctions which are extremely important for the mathematics, but not so important in the cases one deals with in physics.
In physics, you never have enough time, and we must always have a only just precise enough understanding of the mathematics that can be transmitted maximally quickly to the next generation. Often this means that you forsake full rigor, and introduce notational short-cuts and imprecise terminology that makes turning the argument rigorous difficult.
Some of the arguments in physics though are pure magic. For me, the replica trick is the best example. If this ever gets a rigorous version, I will be flabbergasted.
1) What are the most important and the oldest insights (notions, results) from physics that are still lacking rigorous mathematical formulation/proofs.
Here are old problems which could benefit from rigorous analysis:
- Mandelstam's double-dispersion relations: The scattering amplitude for 2 particle to 2 particle scattering can be analytically expanded as an integral over the imaginary discontinuity $\rho(s)$ in the s parameter, and then this discontinuity $\rho(s)$ can be written as an integral over the t parameter, giving a double-discontinuity $\rho(s,t)$ If you go the other way, expand the discontinuity in t first then in s, you get the same function. Why is that? It was argued from perturbation theory by Mandelstam, and there was some work in the 1960s and early 1970s, but it was never solved as far as I know.
- The oldest, dating back centuries: Is the (Newtonian, comet and asteroid free) solar system stable for all time? This is a famous one. Rigorous bounds on where integrability fails will help. The KAM theorem might be the best answer possible, but it doesn't answer the question really, since you don't know whether the planetary perturbations are big enough to lead to instability for 8 planets some big moons, plus sun.
- continuum statistical mechanics: What is a thermodynamic ensemble for a continum field? What is the continuum limit of a statistical distribution? What are the continuous statistical field theories here?
- What are the generic topological solitonic solutions to classical nonlinear field equations? Given a classical equation, how do you find the possible topological solitons? Can they all be generated continuously from given initial data? For a specific example, consider the solar-plasma--- are there localized magneto-hydrodynamic solitons?
There are a bazillion problems here, but my imagination fails.
2) The endeavor of rigorous mathematical explanations, formulations, and proofs for notions and results from physics is mainly taken by mathematicians. What are examples that this endeavor was beneficial to physics itself.
There are a few examples, but I think they are rare:
- Penrose's rigorous proof of the existence of singularities in a closed trapped surface is the canonical example: it was a rigorous argument, derived from Riemannian geometry ideas, and it was extremely important for clarifying what's going on in black holes.
- Quasi-periodic tilings, also associated with Penrose, first arose in Hao and Wang's work in pure logic, where they were able to demonstrate that an appropriate tiling with complicated matching edges could do full computation. The number of tiles were reduced until Penrose gave only 2, and finally physicists discovered quasicrystals. This is spectacular, because here you start in the most esoteric non-physics part of pure mathematics, and you end up at the most hands-on of experimental systems.
- Kac-Moody algebras: These came up in half-mathematics, half early string theory. The results became physical in the 1980s when people started getting interested in group manifold models.
- The ADE classificiation from Lie group theory (and all of Lie group theory) in mathematics is essential in modern physics. Looking back further, Gell-Mann got SU(3) quark symmetry by generalizing isospin in pure mathematics.
- Obstruction theory was essential in understanding how to formulate 3d topological field theories (this was the subject of a recent very interesting question), which have application in the fractional quantum hall effect. This is very abstract mathematics connected to laboratory physics, but only certain simpler parts of the general mathematical machinery are used.
3) What are examples that insisting on rigour delayed progress in physics.
This has happened several times, unfortunately.
- Statistical mechanics: The lack of rigorous proof of Boltzmann ergodicity delayed the acceptance of the idea of statistical equilibrium. The rigorous arguments were faulty--- for example, it is easy to prove that there are no phase transitions in finite volume (since the Boltzmann distribution is analytic), so this was considered a strike against Boltzmann theory, since we see phase transitions. You could also prove all sorts of nonsense about mixing entropy (which was fixed by correctly dealing with classical indistinguishability). Since there was no proof that fields would come to thermal equilibrium, some people believed that blackbody light was not thermal. This delayed acceptance of Planck's theory, and Einstein's. Statistical mechanics was not fully accepted until Onsager's Ising model solution in 1941.
- Path integrals: This is the most notorious example. These were accepted by some physicists immediately in the 1950s, although =the formalism wasn't at all close to complete until Candlin formulated Grassman variables in 1956. Past this point, they could have become standard, but they didn't. The formalism had a bad reputation for giving wrong results, mostly because people were uncomfortable with the lack of rigor, so that they couldn't trust the method. I heard a notable physicist complain in the 1990s that the phase-space path integral (with p and q) couldn't possibly be correct because p and q don't commute, and in the path integral they do because they are classical numbers (no, actually, they don't--- their value in an insertion depends discontinuously on their time order in the proper way). It wasn't until the early 1970s that physicists became completely comfortable with the method, and it took a lot of selling to overcome the resistance.
- Quantum field theory construction: The rigorous methods of the 1960s built up a toolbox of complicated distributional methods and perturbation series resummation which turns out to be the least useful way of looking at the thing. It's now C* algebras and operator valued distributions. The correct path is through the path integral the Wilsonian way, and this is closer to the original point of view of Feynman and Schwinger. But a school of rigorous physicists in the 1960s erected large barriers to entry in field theory work, and progress in field theory was halted for a decade, until rigor was thrown out again in the 1970s. But a proper rigorous formulation of quantum fields is still missing.
In addition to this, there are countless no-go theorems that delayed the discovery of interesting things:
- Time cannot be an operator (Pauli): this delayed the emergence of the path integral particle formulation due to Feynman and Schwinger. Here, the time variable on the particle-path is path-integrated just like anything else.
- Von-Neumann's proof of no-hidden variables: This has a modern descendent in the Kochen Sprecher theorem about entangled sets of qubits. This delayed the Bohm theory, which faced massive resistance at first.
- No charges which transform nontrivially under the Lorentz group(Coleman-Mandula): This theorem had both positive and negative implications. It killed SU(6) theories (good), but it made people miss supersymmetry (bad).
- Quasicrystal order is impossible: This "no go" theorem is the standard proof that periodic order (the general definition of crystals) is restricted to the standard space-groups. This made quasicrystals bunk. The assumption that is violated is the assumption of strict periodicity.
- No supergravity compactifications with chiral fermions (Witten): this theorem assumed manifold compactification, and missed orbifolds of 11d SUGRA, which give rise to the heterotic strings (also Witten, with Horava, so Witten solved the problem).
4) What are examples that solid mathematical understanding of certain issues from physics came from further developements in physics itself. (In particular, I am interested in cases where mathematical rigorous understanding of issues from classical mechanics required quantum mechenics, and also in cases where progress in physics was crucial to rigorous mathematical solutions of questions in mathematics not originated in physics.)
There are several examples here:
- Understanding the adiabatic theorem in classical mechanics (that the action is an adiabatic invariant) came from quantum mechanics, since it was clear that it was the action that needed to be quantized, and this wouldn't make sense without it being adiabatic invariant. I am not sure who proved the adiabatic theorem, but this is exactly what you were asking for--- an insightful classical theorem that came from quantum mechanics (although some decades before modern quantum mechanics)
- The understanding of quantum anomalies came directly from a physical observation (the high rate of neutral pion decay to two photons). Clarifying how this happens through Feynman diagrams, even though a naive argument says it is forbidden led to complete understanding of all anomalous terms in terms of topology. This in turn led to the development of Chern-Simons theory, and the connection with Knot polynomials, discovered by Witten, and earning him a Fields medal.
- Distribution theory originated in Dirac's work to try to give a good foundation for quantum mechanics. The distributional nature of quantum fields was understood by Bohr and Rosenfeld in the 1930s, and the mathematics theory was essentially taken from physics into mathematics. Dirac already defined distributions using test functions, although I don't think he was pedantic about the test-function space properties.
5) The role of rigor is intensly discussed in popular books and blogs. Please supply references (or better annotated references) to academic studies of the role of mathematical rigour in modern physics.
I can't do this, because I don't know any. But for what it's worth, I think it's a bad idea to try to do too much rigor in physics (or even in some parts of mathematics). The basic reason is that rigorous formulations have to be completely standardized in order for the proofs of different authors to fit-together without seams, and this is only possible in very long hindsight, when the best definitions become apparent. In the present, we're always muddling through fog. So there is always a period where different people have slightly different definitions of what they mean, and the proofs don't quite work, and mistakes can happen. This isn't so terrible, so long as the methods are insightful.
The real problem is the massive barrier to entry presented by rigorous definitions. The actual arguments are always much less daunting than the superficial impression you get from reading the proof, because most of the proof is setting up machinery to make the main idea go through. Emphasizing the rigor can put undue emphasis on the machinery rather than the idea.
In physics, you are trying to describe what a natural system is doing, and there is no time to waste in studying sociology. So you can't learn all the machinery the mathematicians standardize on at any one time, you just learn the ideas. The ideas are sufficient for getting on, but they aren't sufficient to convince mathematicians you know what you're talking about (since you have a hard time following the conventions). This is improved by the internet, since the barriers to entry have fallen down dramatically, and there might be a way to merge rigorous and nonrigorous thinking today in ways that were not possible in earlier times.