All bodies radiate heat according the the [black body distribution][1] for the temperature they are at.

This radiation is energy transferred radially away from the bodies , and its magnitude is given by the [Stephan Boltzmann law][2] .  The radiance (watts per square metre per steradian) is proportional to the fourth power of the temperature. Thus a colder body will radiate to the hot body much less than the hot body to the cold. As the hot body will be cooling by its radiation, the rate of cooling will be less than it would be if there were no cold radiator close to it. On the other hand as the incoming radiation from the hot body is much larger than the loss of its black body radiation, the hot body will get heated. 

Maybe  the quantum mechanical picture, or radiation being photons of energy h*nu will help also. Many more photons of higher energy impinge from the hot body to the cold one, than from the cold body to the hot one.




  [1]: https://en.wikipedia.org/wiki/Black-body_radiation
  [2]: https://en.wikipedia.org/wiki/Stefan%E2%80%93Boltzmann_law