How and from where does a mathematician learn physics from a mathematical stand point? I am reading the book by Spivak Elementary Mechanics from a mathematicians view point. The first couple of pages of Lecture 1 of the book summarizes what I intend by physics from a mathematical stand point. I wanted to find out what are the other good sources for other branches of physics.
You want the book by V.I. Arnold, Mathematical Methods of Classical Mechanics. It takes a very rigorous, axiomatic approach to Lagrangian and Hamiltonian mechanics, and it should be accessible to, and enjoyable by, a broad spectrum of mathematicians.
For more details see this review by Ian Sneddon, which also covers Walter Thirring's A course in mathematical physics, vol. 1: Classical dynamical systems.
It is not just local-patriotism, but because I really think this book series should be very accessible to Mathematicians: Walter Thirring, A Course in Mathematical Physics, various volumes.
For more details see this review by Ian Sneddon, which also covers V.I. Arnold's book.
For quantum field theory I like Folland's book "Quantum Field Theory. A Tourist Guide for Mathematicians", because it is written by a mathematician with mathematicians as readers in mind. It is full of comments and explanations that a mathematician needs and are usually not in the physics books. Also he clearly says which parts a rigorous, from a mathematics point of view, and which are not.
For mechanics at the next level (or perhaps skipping a level), you could try Jerrold E. Marsden & Tudor S. Ratiu, "Introduction to Mechanics and Symmetry", Springer, 1994.
For quantum field theory, a recent attempt at a moderately elementary level is http://www.amazon.com/Quantum-Theory-Mathematical-Surveys-Monographs/dp/0821847058, by Gerald B. Folland, which has the subtitle "a tourist guide for mathematicians".
If you're into general relativity, you could try General Relativity for Mathematicians, by Sachs and Wu. I only know general relativity for physicists, so I can't comment on whether this book is any good, but it might be worth a try.
Sachs is the one known to relativists and cosmologists for the Sachs-Wolfe effect. He taught my E&M class when I was a graduate student at Berkeley in the early 1990s. He was housed in the mathematics department, but I guess he had a joint appointment in physics.
I personally don't know of particular books dedicated to the subject covering all areas of physics (maybe "Mathematical Methods for Physics and Engineering" by Riley, Hobson and Bence isn't quite what you're looking for), but if you happen to come across the subject of Quantum Field Theory then I suggest you have a look at "Quantum Field Theory for Mathematicians" by Ticciati.
You might try, now in paperback,
Th. Frankel: The Geometry of Physics, An Introduction, Cambridge U.P. (Cambridge), 1997.
It's a course in differential geometry, actually, but one oriented towards physics, with succinct but comprehensive enough developments of physical theories (mechanics, electromagnetism, thermodynamics, Yang-Mills ...). It's a bit like Burke's Applied Differential Geometry (which I like too), but longer and more systematic.
E. Zeidler, Quantum Field theory I Basics in Mathematics and Physics, Springer 2006. http://www.mis.mpg.de/zeidler/qft.html
is a book I highly recommend. It is the first volume of a sequence, of which not all volumes have been published yet. This volume gives an overview over the main mathematical techniques used in quantum physics, in a way that you cannot find anywhere else.
It is a mix of rigorous mathematics and intuitive explanation, and tries to build ''A bridge between mathematiciands and physicists'' as the subtitle says.
The main difference between mathematicians and physicists is that the former define their terms, and the latter do not. I.e., mathematicians are logical, physicists, even theoretical physicists, except for Dirac, are willing to be illogical. A novel about Oxford life had in it the line «Of course, you began by defining your terms...» so this distinction is relevant in the rest of life as well....
Do not neglect Laurent Schwartz's wonderful book Mathematics for the Physical Sciences. This is not at all what you asked for.
Hertz and Maxwell were both willing to be logical on occasion and both wrote wonderful physics books attemtping to better Newton's great Principia: do not neglect Hertz's The principles of mechanics : presented in a new form or Maxwell's Matter and Motion. This is not quite what you wanted either.
Sommerfeld's five volume series is still the best: he was a great physicist and everything is presented from a physics viewpoint, but he is logical enough to be undertanded of the pe... I mean, the mathematicians. This is what you wanted. I am not pleased with Arnol'd, Marsden, Sternberg, et hoc genus omne. I would recommend giving them a miss. I consult Greiner's volumes quite often, but you had better avoid them. I do have a high opinion of Thirring's series, but I don't know why.... Feynman is not what you want either. For Quantum Mechanics, though, Dirac's famous textbook is indeed what you want, and there is no alternative (umm. well, maybe Vladimir Fock) (it is not included in Sommerfeld's series).
You mention, in specific, GenRel. Read Weyl. (On this topic, at least, Landau and Lifschitz are not trustworthy.) Or else the original papers by Einstein and Hilbert. It is unfortunate that Weyl and Hilbert, in order to justify to themselves publishing something where they felt Einstein obviously had priority, felt it necessary to include «something more» so each one tried to unify gravity with electromagnetism...and neither Mother Nature nor History has judged their «extras» too kindly...--- but Weyl at least invented gauge field theory while trying to do this, so I cannot decide whether or not to tell you to absolutely skip every part of Weyl's book where he tries to «improve» on Einstein. I would like to, but I cannot bring myself to do it.
Mathematics is a language in which physicist express their ideas. It is mathematics which helps to reach and imagine results which are far beyond reach of direct imagination of a physicist mind, at times. Mathematics really helps to imagine complex ideas.
A mathematician has all the tools ready in his hand to learn physics, as physics utilizes power of mathematics to understand nature. Hence, I believe that it would be of really help if mathematician understands how physicist employs math and how physicist see or interpret math for physical concepts.
With classics like Landau's book, Goldstein's book, I would suggest two other classics: Feynman's Lectures on physics part 1 and Berkeley physics course part 1 for classical mechanics.
Feynman's Lectures will be an exact place to learn physicist point of view.