IS heat the only thing keeping CPUs from a faster clock rate? Is heat the only thing that keeps CPUs from having a faster clock cycle or are there other limiting physical factors which keeps CPUs under 3~5GHz? 
 A: No, but yes.
We could mostly eliminate the heating problem by using larger wafers with more space between the transistors, which would permit bigger heatsinks to transport off the heat quicker, thus allowing higher average power per transistor and therefore higher clock speed. Just, it would not really be helpful:


*

*Bigger-space layouts are problematic for fast CPUs. The longer wires entail higher parasitic capacities, to ground and in particular also to other wires, if these run so closely side-by-side as you'd still need even with less-packed layout, i.e. you'd probably get crosstalk problems which would need to be fought with yet more complicated design and possibly requirer slower clock speed again. They'd also add some power consumption, but I reckon the capacity of the FET gates (the transistors themselves) would still dominate.You might even run into sync issues: at those clock frequencies, speed of light doesn't look very fast anymore. During a 10 GHz cycle, the signals travel only $\approx 3\:\mathrm{cm}$; the discrepancies in arrival time between nearby and further away parts of the CPU could easily lead to signal corruption.

*Bigger wafers would mean higher prices. That's not economical if you can use the same space to just add another CPU core!

*Power consumption increases faster than linearly with increased clock speed. So any speed gain of such a larger core could at best very slightly make up for the disadvantages.


Hence, yes, effectively heat is the single limiting factor, preventing us from speeding CPU clocks above 5 GHz.
We might better say though: power consumption is the limiting factor. Heat is not the only reason you care about power: increasingly, the energy bilance is an important performance characteristic of a CPU, most obviously in the mobile sector but also in datacenters / supercomputers. And mostly, you get the lowest energy consumption per calculation by using many small processors running at relatively low clock speed.
A: Fundamental limitation of CPU speed is speed of light - and hence this question belong to this site :-) 
Electrical signal in wires travel typically 15cm per nanosecond (~0.5c). 
Large CPU could have die size of ~15x15mm. You can see that to travel corner to corner will take ~0.14ns, 0.28ns for a round trip. So you can see that our rough limitation of speed is already at ... 3.57Ghz. 
Surely there is no way around speed of light, other than making things smaller - but this is no longer helping as we are indeed getting limited by power density (exceeding one in core of nuclear reactor).
3D chips will make interconnections much shorter, but problem of heat dissipation will get worse. 
After all that - there are inductance/capacicance/resistances around. The smaller wires you have - the larger their resistance gets. But it's probably not the worst part of the problem. 
On the good side - engineers used to work with "slow" propagation of signals   using pipelined designs and massively parallel computation. 
