The successful use of Yang-Mills theory to describe the strong interactions of elementary particles...
Who says it's successful? The nuclear force remains one of the unsolved problems in physics: "What is the nature of the nuclear force that binds protons and neutrons into stable nuclei and rare isotopes? What is the origin of simple patterns in complex nuclei?" Forces between quarks and gluons doesn't make up for that. Especially when the gluons in ordinary hadrons are virtual, and we've never ever seen a free quark.
depends on a subtle quantum mechanical property called the "mass gap" as we know: the quantum particles have positive masses, even though the classical waves travel at the speed of light.
IMHO quantum particles such as the electron have a positive mass whilst electromagnetic waves are massless because of E=mc². As Einstein said, "the mass of a body is a measure of its energy-content". When you trap a massless photon in a mirror-box, you increase the mass of that system. When you open the box, it's a radiating body that loses mass. See https://arxiv.org/abs/1508.06478 by van der Mark and (not the Nobel) 't Hooft. Think of photon momentum as resistance to change of motion for a wave moving linearly at c. Think of the extra mass of the mirror-box as resistance to change of motion for a wave going round and round at c. As for the mass gap, see Wikipedia:
"the mass gap is the difference in energy between the lowest energy state, the vacuum, and the next lowest energy state. The energy of the vacuum is zero by definition and assuming that all energy states can be thought of as particles in plane-waves, the mass gap is the mass of the lightest".
It's not a good idea to make assumptions in physics. An electron is a spinor. We can diffract electrons. The wave nature of matter is not in doubt. But an electron isn't going past you at c like a photon. Check out Hans Ohanian’s 1984 paper what is spin? Pair production works the way that it does and the electron has the mass it has for a good reason. However that isn't part of the standard model.
Now, my question is, this property has been discovered experimentally and computationally; but how can it be understood from a theoretical point of view?
By looking outside the standard model. In particular I'd say it's important to appreciate that space has particular properties associated with E=hf electromagnetic waves. And that in atomic orbitals electrons exist as standing waves. So the mass gap concerns the way in pair production you form an electromagnetic wave into a spin ½ standing-wave spinor called the electron. The standard model doesn't tell you anything about this, so you end up in something of a catch-22 situation. To understand something in the standard model, you have to understand something that isn't in the standard model, and is arguably in conflict with it.