I'm not sure whether it's best to resurrect this old question, or pose a new one, but here goes:
Is it not the case that the sensor emits IR toward the scene, just as the scene emits IR toward the sensor?
If the sensor is viewing a scene at a lower temperature, that cools the sensor. The sensor emits more energy toward the scene than the scene emits toward the sensor.
The camera body and lens have low emissivity, so they don't radiate enough thermal energy to drown out this effect. The sensor has high emissivity -- it must, because it needs to absorb incoming IR with high efficiency, instead of reflecting it away.
This same reasoning works if you consider an individual sensor pixel, and the corresponding small patch of the scene. That's how a thermal camera can image objects colder than itself.
This is also how the ground cools below air temperature on a clear night. The sky looks very cold, and as the ground radiates heat toward it, the sky doesn't radiate nearly as much heat back. The result: the ground can get cold enough to accumulate frost, even when the air temperature is above freezing, all without breaking a single law of thermodynamics!