100% quantum efficiency does not translate to 100% efficiency. It simply means 'one electron out per photon in'.
A standard single-junction cell has a maximum thermodynamic efficiency of ~33% at 100% quantum efficiency and a bandgap of ~1.34eV. (The maximum for single-junction silicon cells is slightly worse, at ~32% and 1.1eV.)
Why the difference? Solar radiation is a spectrum. If you have a solar cell with a bandgap of 1.34eV, an incoming photon with less energy will be 'wasted' (won't produce useful output energy), and an incoming photon with more energy... will still only produce 1.34eV of useful energy. So there's a tradeoff - higher bandgaps mean each absorbed photon produces more useful energy, but more photons are 'wasted' and don't produce any useful energy. 1.34eV is the optimum here.
Well, what about something like your design? Something that splits into an (near)infinite series of different cells optimized for specific wavelengths.
Let's simplify for a moment. Instead of a giant splitter, let's just imagine sunlight aimed through a material that passes a narrow window at 500nm and reflects anything else, at a 2.5eV bandgap solar cell. (Assume that the solar cell is illuminated like this from all directions.)
Well... as it turns out that's still not a free lunch. See, we've been talking about solar cells as an irreversible process. Photons come in, useful energy comes out. But this is actually a reversible process. It's actually more like photons kick electrons, forming electron-hole pairs. These pairs then wander around, potentially dissipating energy (in the form of heat) as they cross the PN junction and either recombine (forming more photons), or escape the solar cell (producing useful energy).
You can view this system as having a sort of 'effective' temperature. A high temperature corresponding to many electrons and holes wandering about, and a low temperature corresponding to few. Short-circuit operation corresponds to ambient temperature, and open-circuit operation corresponds to the temperature of the sun (or whichever input source).
Interestingly, when you work out the overall efficiency, comparing energy flows and useful output work, you get the same as a Carnot engine working between said temperature and a cold side consisting of the solar cell.
...which in turn means that your maximum possible efficiency is at 'nearly' open-circuit operation, and at (near) Carnot efficiency between the sun and solar cell ambient temperature.
And of course, splitting the input into an series of engines each operating at no more than Carnot efficiency, itself operates at no more than Carnot efficiency.
(Note that this is not the maximum output power point. Near-open-circuit operation results in near-zero output power. The maximum power point for said stack of monochromatic cells is at ~87% efficiency for 'standard' conditions.)
For more details and the backing calculations, see "Solar cell as a quantum converter" in Theoretical Calculation
of the Efficiency Limit for Solar Cells here.