It can be argued that there are two primary challenges associated with quantum computing. One is a coding challenge, and the other is a purely mechanical (or in some minds physical) challenge.
As Ron indicated, viable Quantum Error Correction Code (QECC) is probably the largest breakthrough that makes quantum computers possible. Peter Shor was the first to demonstrate a viable Quantum Error Correction Code that can also be characterized as a Decoherence Free Subspace.
At this point, it can be argued that most of the problems associated with Fault Tolerant Quantum Computing are resolvable in terms of theory. However, their is a question of scaling as it relates to the number of qubits that are required to make the system viable, and the physical size of the system that is required to support those qubits. Most arguments that say there is a needed breakthrough in physics can be traced to this question of scalability.
In many cases, the difficulties are related to decoherence times and the speed at which the computers can perform calculations, which would allow the QECC time to operate. At this point those are now considered largely engineering challanges.
As far as physical breakthroughs, it is likely that those conversations are in regards to Topological Quantum Computers, which rely upon the construction of Anyons in order to operate. Currently these are largely viewed as pure mathematical constructions, however, the construction and use of stable anyons, even as quasiparticles, as part of topological quantum computer would be a major physical breakthrough.