Say you have a scaled up system with 98% fidelity so 2% decoheres. These are physical qubits doing error correction on information encoded on logical qubits. 98% of these entangled qubits will be doing error correction but 2% will interact with the environment and decohere. Since this 2% carries a small part of the coded information from the quantum gate where they were doing error correction, what happens to those physical qubits when they decohere? Is it just dead code?
I take it that you are using a short-hand way to describe the true situation, in which everything decoheres a little but we can analyse this as if some things decohere and some things don't.
If the physical bits you are referring to are part of the basic level in the ancilliary aparatus employed in the error correction, then what happens is that their quantum state becomes uncorrelated with the state of the rest of the computer, such that whatever role they were intended to play in the next corrective step will not take place or will go wrong. In a typical design, these bits will then be re-prepared in any case in some standard state such as the zero state, and then they enter the next round of correction all fresh and new and error free (but of course even this repreparation will not be completely perfect).
If these bits were somewhere else in the heirarchy, then they may be corrected by some of the further apparatus. In either case the noise they caused by going wrong will, most likely, be corrected in the next round, but of course at every stage a little bit of noise leaks through and slowly builds. One designs the whole protocol such that this "little bit" is small compared to $QN$ where $Q$ is the number of logical steps and $N$ the number of logical qubits in the algorithm being computed.