There appears to be a confusion here regarding the idea of absorption. In one sense, every photon which interacts with an object or medium is “absorbed”. Some of that absorbed energy is re-emitted, generating the reflected (or transmitted) light, usually of the same color. But some of the energy is not re-emitted; it is lost, converted into another form such as heat. This lost energy is often called the “absorbed” light, in the second sense that you referred to. So indeed, the color of an object is due to the light energy that was not lost, or absorbed, in the object.
To summarize, in the “energy is lost” sense, you are right, the light is not absorbed in the kind of reflection you would normally think of. But in the sense that every photon which reflects from an object is absorbed and subsequently re-emitted, then the light is “absorbed” during the interaction. This is the source of the confusion: there are two concepts of “absorption”. One refers to absorption as a necessary step of the reflection interaction, and the other refers to absorption as a loss of energy into the object.