Resources that provide physical insight into quantum mechanics

Question

So I've took a first undergrad quantum mechanics course and found it to be very dissatisfactory to say the least. We covered roughly the same material as classical textbook like Griffiths. I understand the abstract mathematical formulation of elementary quantum mechanics and find it very interesting as a mathematical theory. I find it very exciting that abstract mathematical topics like spectral theory of unbounded operators or distribution theory have concrete applications in describing the world we live in.

However, I don't really see why this whole quantum mechanics business is actual physics and helps us describe the "real world". Every quantum mechanics textbook (and the course I took) I looked at had the same structure: First, the need for a new theory beyond classical mechanics is presented by examples like the ultraviolet catastrophe, the electron falling into the nucleus, the Compton scattering and so on. This is fine. Then for me the disconnect happens: The abstract formulation of quantum mechanics modelled by Hilbert spaces and operators acting on them is "introduced" and the author goes on a rampage calculating different theoretical models like the particles in various potentials or finding eigenvalues of pertubed Hamilton operators. In this nightmare of calculations various phenomena like discrete energy values, tunneling or wave functions of entangled particles show up and through all of this one is supposed to "understand quantum mechanics". However, for me all of this provides little insight into why we expect this abstract formulation of quantum mechanics to describe reality in some way in the first place.

I understand that ultimatively a physical theory is just a tool describing reality (whatever that is) and if the theoretical predictions match the experimental results closely enough, the theory is considered "valid". But physics seems to me to be much more than using the tools some genius like Schrödinger or Dirac came up with and using them to separate the Schrödinger equation of a particle in some fictional potential. I want to know why and how they came up with these theories in the first place and what the physical intuition behind them is.

Sorry if this comes across as a bit of a rant on physics pedagogy, but I think this helps to understand better what sort of literature I'm looking for. I'm looking for a book that aims to give som insight into the physical intuition behind the abstract formulation of quantum mechanics. I can imagine two different interesting approaches:

• Following the historical development of quantum mechanics and explaining what various problems the physicists involved had to solve and why they did it the way they did (I mean Dirac won't have woken up one night and thought "Physical observables correspond to unbounded self-adjoint operators on a separable complete inner product space over the complex numbers!").
• Comparing the results of various real-world experiments with the results given by theoretical calculations and explaining why the formulation and axioms of quantum mechanics have to be chosen in exactly this way to match the experimental results.

I hope the text above helps in describing what sort of text I'm looking for.

Answer 1

I sympathise with your frustration. I was naturally good at physics before I went to university (often scoring 100% in mock entrance tests etc), I think because I developed very strong mental pictures of what the physical theories represented, so strong that I could usually work out most problems from first principles. That all changed at university, where the lecturers seemed to be regurgitating words and formulae without having any understanding of the underlying reality. It was like learning history- you were just expected to memorise stuff. I think the standard of physics teaching is poor, generally, and most textbooks spend far too much time on rigorous mathematics and far too little on ensuring the reader understands the principles that the mathematics is modelling.

The areas that seem worst taught are QM and special relativity. That is reflected in the number of questions (and erroneous answers) about those topics on this forum.

I have spent many years trying to develop a coherent conceptual model of QM, which is tricky because of the inherent complexity of the topic and the high mathematical content, but I will do my best to answer any specific questions you might have. In the meantime there is a blog that introduces the ideas of quantum mechanics without any mathematics, which you can find here http://simplequantumtheory.blogspot.com You need to read the posts in chronological order (ie starting from the oldest).

Answer 2

I highly recommend that you check out the third volume of the Feynman lectures. Feynman eschews heavy mathematical formalism and develops quantum mechanics by using it to explain various experimental results. This should help you connect the math to its real-life experimental consequences.

I also recommend the first chapter of Weinberg's Lectures on Quantum Mechanics, which gives a historical introduction to the subject. Weinberg walks step-by-step through the mathematical developments and explains how they grew out of the experimental results.