The relevant form of adiabatic expansion comprises two components. One is the expanding gas and the second is the system that opposes the expansion, without which the expansion would not perform pdV$pdV$ work. The relation dE=-pdV$dE=-pdV$, describes the transfer of energy to the second system. In the case of the universe, gravity opposes the expansion of the photon gas. The expanding gas of photons loses energy while the universe gains the work energy.
The previous answer describes an expanding box of photons doing work on the rest of the universe which suggests that the pdV energy might be transferred out of the box. However, this is forbidden because the universe is isotropic and homogeneous. If there is any net energy transfer out of an expanding volume, by the homogeneity assumption there must be equal and opposite energy transfer out of an immediately adjacent volume, therefore there cannot be any net energy transfer out of any volume of the universe. The pdV energy must remain within the expanding volume.
The universe is governed by the Friedmann equations. The pdV$pdV$ energy must be found in one of the parameters of the Friedmann equations, which are the energy density and pressure of the components, the curvature parameter, and the cosmological constant. Since experiments show that the curvature and the cosmological constant are negligible at the time of photon decoupling, the only possible manifestation of the pdV energy is as an energy density.
The relation dE=-pdV$dE=-pdV$ can be derived directly from the Friedmann equations. Hence neglecting the pdV energy is in contradiction to the Friedmann equations. The conclusion is that the lost photon energy must be included as an additional energy density in any model of the universe.