Yes, information in its basic simplest form, in quantum theory, is the state of the system (which could be composed of many subsystems). A physical system is defined by a state vector. It could and often is infinite dimensional, but could also have finite dimensional Hilbert subspaces (like the spin). The evolution of a system,considered a pure state, is given by a unitary operator which preserves causality (at the Hilbert space level, not in the probabilistic interpretation of collapse and measurements). You can always go back by applying the inverse operator. When the state becomes mixed information can be considered to be lost, and entropy increases.
The preservation of information is thought, in this way of describing it, to be equivalent to the unitary evolution of a system.
The problem that arose with Black Holes (BH), the No Hair Theorem and the Hawking radiation from a BH which is thermal (i.e., no information) is that as matter falls into the BH, say a pure electron, the BH keeps no trace of it, and as it the BH evaporates no state that was the unitary evolution of the electron state remains, i.e. information is lost. There is an unresolved paradox, the BH information paradox, with partial but not totally resolved solution. The most recent thought is that the information remains in the horizon as some kinds of quantum hair, and there is entanglements between particles being radiated as Hawking radiation and somehow it does not get lost, nothing fully shown but somewhat backed up by the AdS/CFT correspondence (gravity in the bulk is equivalent to a CFT on its surface).
The information loss paradox describes the paradox, but also explains the meaning of information (basically the unitarity of the state evolution in quantum theory), why information is thus assumed to be conserved, and why it would violate all kinds of physics if it is not. See a simple summary at https://en.m.wikipedia.org/wiki/Black_hole_information_paradox
There are plenty of articles and even books on it. There was a bet between Hawking and Susskind as to whether information was lost or gravity was wrong. Hawking paid up, but it's all still is not perfectly clear, and won't be till there is a quantum theory of gravity.
