One possible reason I have come up with is that we are modeling quantum gates by unitary matrices. And since unitary operations are reversible we have to be able reverse the operation in the physical world as well. This is simply done by "remembering" which state has gone in and provide it as output if the reverse operation should be applied.
But what happens if we don't "remember" the input state? Are there certain operations that can be performed without the need of reversibility?