A Sizable Mystery
Here's a mystery that remains poorly understood, though there have been many attempts to explain it:
Why does volume -- the ability of matter to fill up space exclusively -- depend on how particles rotate?
By volume I mean for example the fact that you can pound on a desk with your fist, and your fist stops at the desk. The matter in your desk and your fist exclude each other from occupying the same space. Without volume, the universe would be a very boring place. That's because instead of planets, suns, and nebulae we would have black holes, black holes, and black holes. Furthermore, the same features that enable volume also enable all the incredible richness and variety of combination called chemistry. So, without the physics of volume, we would not be here to talk about the topic in the first place.
For Every Volume, Turn, Turn, Turn
Yet the existence of volume depends rather remarkably on the way some particles rotate. It is that simple connection to rotation that remains mysterious and still smells of something important being overlooked, of some insight that if finally found would make everyone go "Ah! So that's what's really going on there!" But that simple insight remains missing, even though folks such as Nobel Laureate Richard Feynman worked on the problem off and on for decades, without any notable success.
I should emphasize first that how volume works is very well understood.
An Exclusive Club
It's created by something called the Pauli exclusion principle, which behaves like an extremely powerful repulsive force that only comes into effect when identical particles of a certain type, called fermions, are pressed closely together. Fermions are what we usually think of as matter, and they have an "address space" with three parts: location, momentum, and spin orientation (think axis of a spinning globe). As long as all particles remain unique in at least one part of this address space, the fermions are happy, which is to say they stay fairly low in energy. All of the geometry and bonding mechanisms of chemistry arise directly from the rather complicated interplay of a nucleus that attracts a set of electrons, and of all of those electrons insisting on having their own unique three-component addresses.
But that is the known part. The hard-to-explain-well part is why Pauli exclusion is experimentally tied to a very specific type of particle rotation.
As with many quantities in quantum mechanics, the rotation of a very small object begins to lock into discrete values that are based on their angular momentum. Realizing that this quantization would have to occur for angular momentum, physicists defined the smallest unit of angular momentum as spin 1. No one really thought much about it at first, since spin just seemed like yet another "feature" that needed to be tracked when talking about atoms and particles.
A Tale of Two Particle Types
This assumption turned out to be spectacularly wrong. It was subsequently realized from experimental data that the entire universe seems to break down into two major classes of particles, and that these two classes are based entirely on how they spin. The first group is the fermions I've already talked about; they are the ones that have Pauli Exclusion and thus volume.
The second group is called the bosons. The bosons have spins that are simple integer multiples of the smallest obvious unit of quantized spin, spin 1. But these fundamental particles have no volume! They not only don't care at all if they share the same address, there are cases where they prefer to have the same address. That is what a laser is: A lot of spin 1 particles of light that have decided to join together and all occupy the same location, momentum, and spin address at one time. Fundamental bosons are what we usually think of as some form of energy.
But if bosons have rotations that are simple multiples of the smallest possible unit of rotation, spin 1, what kind of rotation can fermions have that is different? Where do they fit in?
The Sound of Half a Rope Spinning
That is the first really weird thing about volume: It is based on particles whose rotations are offset by exactly $\frac{1}{2}$ unit from the boson rotation values, and thus fit "in between" the integer spin values of the bosons. So for example, fundamental electrons and the more complicated protons and neutrons of matter all have spin $\frac{1}{2}$, and so all occupy space.
If all that sounds odd, it is. The fermion offset of "spin $\frac{1}{2}$" was completely unanticipated to theorists. It was first a source of amusement and then bafflement when experimentation first forced theorist to consider its existence. For a theorist of that time (or now!), trying to interpret the visual meaning of "one half" of the already smallest-possible spin 1 was like trying to visualize one-half of a skip rope loop. After all, in a skip rope you can have one loop, or two, or even more with expert skip rope twirlers -- but less than one loop? What does that even mean?
So just how mysterious is this half-unit of spin?
Reluctantly at First, He Took it For a Spin
Well, Wolfgang Pauli was easily one of the most brilliant (and abrasive) members of the very elite club of physicists who in mid 1920s developed the foundations of modern quantum mechanics. Pauli at first rejected even the idea that point-like electrons could spin, and likely cost Ralph Kronig a Nobel prize because of it. Pauli chastising Kronig so severely just for bringing up the idea that Kronig thereafter argued adamantly against his own idea! Pauli on the other hand subsequently not only repented of his initial view, but ended up developing the mathematical model for spin $\frac{1}{2}$ that is used to this day. The model is called the Pauli spin matrices.
But even someone as intimate with the issue of spin as Pauli pretty much gave up on any kind of conventional explanation of it. Instead, he declared particle spin to be an "abstract property" (p.3, line 9 from bottom) that has no particular connection to ordinary rotation. However, since quantum spin is a just a quantized version of everyday rotation, it is unavoidably deeply linked to it. Thus a more accurate translation of the word "abstract" in this particular context might be: "The math works beautifully, so please just use it and stop asking me what it means!"
So in summary, matter (which mostly likes to stay put, has volume, and resists compression) is built up from fermions whose rotations all have odd spin $\frac{1}{2}$ offsets in their rotations, while energy (which most often is literally as fluid and ephemeral as light and sound, and which can be compressed or focused almost without limit into a small volume) is built up from bosons whose rotations are all multiples of spin 1.
Lies, Darned Lies, and Spin Statistics
The spin statistics theorem is the formal name for all of that, stating that particles with spin $\frac{1}{2}$ are subject to Pauli Exclusion ("volume"), while particles with simple integer spin (or zero spin) are not subject to it. This theorem is primarily a summary of experimental findings; it is not some kind of mathematical result from which fermions and bosons were predicted based on first principles.
And that is why the connection between volume -- the resistance of matter particles to being compressed -- and the spin $\frac{1}{2}$ offset of fermions remains more a mystery than well-understood principle of physics. The proofs devised for it remain unconvincing even to the experts. For example, a 1998 assessment of spin statistics theory by Ian Duck and E.C.G. Sudarshan provides a detailed summary of the strategies theorists have used in trying to prove the spin statistics theorem, yet it concludes with this final line:
"Finally we are forced to conclude that although the Spin Statistics Theorem is simply stated, it is by no means simply understood or simply proved."
Two examples of such proofs include a very early (and still persuasive) [proof by Julian Schwinger, and this much more recent 2003 theory by Paul O'Hara.
Invisible Hands, But Not the Adam Smith Type
One reason why I do not find any of these proofs particularly persuasive is this: If the theorists who created them did not know in advance exactly where they needed to go, it appears unlikely they ever would have managed to arrive at their destination. That situation is in sharp contrast to Paul Dirac's Dirac equation, which remains the gold standard for experimentally predictive theoretical mathematics. Once he came up with it, the Dirac equation pretty much had to drag Dirac kicking and screaming into acknowledging that there must be an entire second universe of antiparticles that are mirror images of regular particles.
The Conclusion
So, while various methods used to prove the spin statistics theorem may well be correct, they feel more like someone forging a circuitous path through deep woods to make their way finally to a bright light they could see off in the distance at all times. It seems quite likely that the main road, the easy path that shows you exactly where that destination lies, has yet to be uncovered. A truly simple explanation of why spin $\frac{1}{2}$ offsets lead to Pauli exclusion, and to its simple, everyday consequence that two objects cannot occupy the same space at the same time, has yet to be found.
