Just like different materials have varying reflectivity that leads to some of the variations we can see in visible images. Different materials have varying emissivity that affects how much they radiate. You end up with less information from the scene than you would have in the corresponding visible image because you dont get all the shadows that your brain uses (in part) to perceive depth.
Also, not all materials are Lambertian and therefore you will see different radiation depending on angle.
Finally, thermal cameras are quite sensitive. Without a lot of effort in establishing uniform temperature in a scene different things and materials will naturally be at different temperatures. This may occur if things are heated/cooled unevenly or if there are differences in absorptivity.