The wikipedia article on color can clear the confusion on color perception. The electromagnetic spectrum in the visible range corresponds to colors as the human eye observes them, but color perception is more general and depends on the phsyiology of the eye and brain. One can make color photographs using for illumination only two frequencies, for example, as polaroid inventor E.Land demonstrated.
A hot iron rod is red because the upper end of the infrared spectrum, which is the heat you are feeling/measuring , not seeing, goes into our perception of red. The overwhelming majority of the energy is in the infrared for which we have no retina sensors, only skin ones.
A book may be red because it is reflecting the red frequency, or we may perceive it as red because of the cones in our retina ( read the article of color in wiki). The cover absorbs the non reflected part of the spectrum but that is a very small amount of energy to be converted to heat from room light, so we do not perceive the difference in temperature.