In the literature of the quantum eraser experiment it is argued that the change in statistics of the system from non-interference to interference is due to the erasing of "distinguishing information". The preparation of the experiment almost always introduces and then erases this information via a pair of unitary actions on the system. In the specific case that the interference effect is a second order effect of single photon interference, by destroying/creating the interference one is causing a physical disturbance to the system in that it changes the measurable statistics of the system, it destroys/creates the interference pattern that is collected at a single detector/screen. If it is distinguishing information that causes this physical change, if this physical change happens in the absence of a physical cause, is it not the case that this is a violation of the law of conservation of energy? There is no "driving force". It might only be the configuration of the statistics of the system that changes (from interference to non-interference or vica versa), but certainly this is a physical change that is measurable, so it should require energy and momentum to "push" the statistics in the right direction(away from or towards constructive interference). Distinguishing information does not provide this driving force.
And I know the generalized set of rules as to why quantum mechanics is suppose to be energy and momentum conserving, it has to do with the invariance of H under dislacements. This does not explain the specific case of quantum erasure which seemingly defies conservation.