I know black conducts heat while white reflects it.
Black objects don't "conduct heat". Black objects absorb incoming radiation in the visible range. Similarly, white objects don't reflect heat. They diffusely reflect incoming visible radiation.
But they are colors after all.
Yes and no. Whether black or white are "colors" depends very much on what you mean by color. I'll leave that debate for a different question. For the purpose of this question, it's better to look at black and white as shades of gray rather than as colors like red and blue.
What is the physics behind this?
This is the question I'll address in detail. The answer lies in the concepts of emissivity, absorptivity, reflectivity, and transmissivity.
- Emissivity is the ability of an object to emit thermal radiation, relative to that of an ideal black body.
- Absorptivity is the fraction of incoming radiation absorbed by an object.
- Reflectivity is the fraction of incoming radiation reflected by an object.
- Transmissivity is the fraction of incoming radiation that passes through an object
The latter three (absorptivity, reflectivity, and transmissivity) completely enumerate what happens to incoming radiation. They add to 1 (or to 100% if you want at those as percentages). For the rest of this answer, I'll assume opaque objects, where transmissivity is zero. Incoming light for opaque objects is either absorbed or reflected, in the ratio determined by the object's absorptivity and reflectivity (which add to one).
Reflectivity and absorptivity explains in part why black objects get hotter than do white ones. A perfectly black object absorbs all incoming visible radiation, while a perfectly white object reflects all incoming visible radiation. As there is no such thing as a perfectly black or perfectly white object, all objects absorb incoming visible radiation to some extent. However, black objects absorb considerably more incoming visible radiation than do white ones.
The flip side of the coin is emissivity. An object will eventually come to thermal equilibrium, with the energy absorbed from incoming radiation being equal to the energy emitted as outgoing radiation. The outgoing radiation is a function of the object's emissivity $\epsilon$, it's temperature $T$, and it's surface area $A$, dictated by the Stefan Boltzmann equation $E_{\text{out}} = A \epsilon \sigma T^4$ where $\sigma$ is the Stefan Boltzmann constant. The incoming radiation is a function of the incoming energy flux $\phi$, the object's absorptivity $\alpha$ and it's cross section to the incoming radiation $A_c$: $E_{\text{in}} = A_c \alpha \phi$. Equating and solving for temperature yields $T= \left( \frac {\alpha}{\epsilon} \frac{A}{A_c} \frac{\phi}{\sigma} \right)^{1/4}$.
Note that only the first factor in the above, $\frac {\alpha}{\epsilon}$ depends on composition. The other two factors represent geometry ($\frac A {A_c}$) and incoming energy ($\frac {\phi}{\sigma}$). Per Kirchhoff's Radiation Law, emissivity and absorptivity at any given frequency are equal. For an ideal gray body, both absorptivity and emissivity are constant, independent of frequency and temperature. The ratio $\frac {\alpha}{\epsilon}$ is one for a perfect gray body. All perfect gray bodies with the same geometry and subject to the same incoming radiation will eventually reach the same equilibrium temperature.
So we need something else to explain why black objects get hotter than do white ones. The answer lies in the fact that absorptivity and emissivity are frequency and temperature dependent for real objects. Ideal gray bodies don't exist. They're nice approximations if applicable. "Black" and "white" refer to the reflectivity (and hence absorptivity) in the visible range. An object that is white visibly can be very black in the thermal infrared. An object that is visibly white but thermally black won't heat up as much as will an object that is visibly and thermally black.