Information on the Photoelectric Effect: https://en.wikipedia.org/wiki/Photoelectric_effect
Summary: Some metals release electrons when struck by a certain frequency of photon.
What does this mean in the context of this question?
Assume that you have found the right metal that will release photons when struck by infrared light. It would seem that you would be able to have completely passive cooling within a house, by setting up panels of this materials to absorb heat (and perhaps generate energy, as a bonus).
What material would you likely need? Would it be cost-effective? Is this even possible for consumer use?