According to quantum mechanics, the electromagnetic radiation is quantized. This mean that it travels in "packets" called quanta. The size of this packet is proportional to the frequency of radiation, higher frequency radiation is carried in larger packets.
So the power delivered by a low frequency radiation comes in a great number of small packets, while the same amount of power carried by an higher frequency radiations is composed by few large packets.
This is the reason why high frequency radiation is more harmful than lower frequency radiation.
If you are irradiated by a low frequency wave is like to be hitted by a great number table tenis ball while an high frequency wave is like be hit by very few bullets.
In particular, high frequency radiation has a ionizing power, which means is able to remove electrons for atoms and eventually damege biological structures like the DNA. Moreover, as the frequency get higher the penetration power of the radiation increase which means that damage can be affect deeper parts of tissues.
Obviously you have to define which is the frequency where harmful effects began to appear. In general, UV light can cause superficial damage (which can however be quite dangerous because skin cancer is a superficial damage, too). Xrays and gamma rays are very dangerous and can cause terrible damage to biological structures.
As far as I know, there are no scientific evidence of damage caused by radiation which frequency is below the visible light.
Obviously the one I described is only the main mechanism of interaction, but is not the only one. Microwaves, for example, are not dangerous in general, but as you can see every time you use your oven, using a specific frequency can cause effect like heating which eventually can cause damage.