The simple answer is that visible light does not correlate with temperature as well as infra-red light does.
The EM waves received by the camera are a mixture of radiated waves and reflected waves from ambient sources. For example, consider your laptop on your desk in the daylight- it will be emitting black-body radiation and reflecting black-body radiation from the sun. Since the black-body spectrum from the sun is dominated by light in the visible range, the overall radiation from your lap-top is predominantly visible, and its intensity is more influenced by the colour of your laptop than by its temperature.
If you want to take an image that correlates better with temperature you have to filter out the EM radiation that is most strongly represented by reflected visible light, ie the radiation that represents the colour of objects rather than their temperature.
If you want to consider a much more extreme example of the same principle, imagine a radio transmitter, a camera and a radio. Both the camera and the radio receive EM radiation from the transmitter. The camera is sensitive to the visible radiation given off by the transmitter, and is useless for detecting the radio signal. The radio is very good at picking up the radio signal, but cannot take a photograph. A device that created an 'image' that was a mix of the visible and radio radiation strength might be an interesting toy to play with!