Cellular signal is transmitted by electromagnetic radiation in the UHF range (ultra high frequency) which is in between radio waves and microwaves. The wavelength is between a decimeter and a meter. Recall that microwaves may be a bit shorter. Infrared lights are even shorter. Visible light is between 1/3 and 2/3 of a micron. Infrared waves are shorter and they're followed by X-rays and gamma rays.
The UHF wavelength not far from a meter is such a long wavelength and, equivalently, the magnetic and electric fields in the wave are changing so slowly that for all atoms, molecules, or cells, or similarly small objects, the wave looks pretty much infinitely long and all effects may be understood by an analysis of a slowly changing static field.
Any effect of this electromagnetic radiation on cells etc. may be summarized as thermal radiation. So just like a (very weak) microwave oven, the cellphone may at most heat the tissues and a part of the brain that is close enough to it.
If you don't feel the heat because the power is just too small, it means that there is no effect. If you managed to find a very strong cell phone and feel some heat somewhere near your ear, the effects of the heat would be exactly as harmless as if the heat came from the Sun (in fact, less harmful) or from a warm mug with coffee. Or someone's warm hand. Clearly, the temporary contact with someone's warm hand isn't harmful, lethal, or dangerous, and the very same comment applies to the microwave or UHF radiation because it is really the same thing.
The UHF and microwave radiation is invisible and some people like to irrationally imply that "the invisible radiation" is always dangerous because the gamma radiation from radioactivity belongs to that class. But it ain't the case. Some invisible radiation is dangerous. Some invisible radiation is harmless and in fact, more harmless than the visible light. That's the case of the cellular radiation frequencies, too.
The X-ray or gamma radiation whose wavelength is significantly shorter than half a micron, the visible (yellow) light's wavelength, is dangerous for individual molecules or cells because it carries enough energy ($E=hf$) in each photon that these photons are able to ionize the atoms or molecules and cause internal changes, like mutation of the DNA. But the microwave, infrared, and UHF/radio waves have so low energy in the photons that this effect is impossible by energy conservation. So cells are at most happily surfing on these long cellular waves, much like a surfer on a peaceful ocean with waves that go up and down every minute. Such ocean waves may be pleasant but they don't really threaten the (good) surfer's stability.
All pages, sources, books, articles that claim that the statements that the cellphone radiation is dangerous are cases of pseudoscience. Sometimes they mislead the readers deliberately, sometimes because of ignorance of basic undergraduate physics.
Sperm prefers to reside at lower temperatures than the temperature of the body. That's why Nature invented the scrotum that isolates the testicles from the rest of the body. Warm pants etc. are not good for the health of these reproductive cells. The heat from the cell phone may have a similar effect. In principle, one could speculate that the sperms try to remember the electromagnetic field for a nanosecond and that could affect their behavior – probably in a reversible way. That can't be completely excluded but I think that it's unlikely because there's no evolution advantage in this ability of the sperm cells, assuming that the environment of the vagina isn't emitting some useful UHF signals. ;-)