The main difficulty (at least to me) lies in the fact that the stuff one has to quantize in general relativity is spacetime itself, while in the other three cases (the other three forces), it are fields in spacetime that are quantized.
You can say that the condensate of virtual gravitons looks the same as a metric tensor (like the condensate of virtual photons looks like the classical electromagnetic field) but still the difficulty remains how to couple these gravitons to the curvature of spacetime. They travel through flat spacetime but how the curvature of spacetime is influenced by the gravitons?
When two particles interact by exchanging gravitons, then the particles themselves absorb (or emit) the gravitons. Or when two massive bodies like the Sun and the Earth move through spacetime the condensate of gravitons is exchanged between the constituents of the Earth and the Sun. To say that the gravitons affect spacetime itself is another thing. So a condensate of gravitons indeed looks like a metric tensor, due to the tensor Nature of the gravitons.
When the particles have a very high kinetic wrt eachother they will form black holes upon collision. Which is another difference between normal quantized fields and gravity quantized "fields". You can incorporate the emergence of black holes into the field theories of colliding particles as one of the possible outcomes of the collision but how the gravitons turn (flat) spacetime into black hole is another thing.
You can of course say that the gravitons are quantized spacetime variations. But then they are non-pointlike structures.
In string theory gravitons are considered to be tiny vibrating closed strings (so non-point-like structures) travelling through a flat background spacetime which is why the theory is said to be background dependent. Are the closed strings quantized spacetime variations? Well, they are extended and they do carry tensor information. But how is this information given to the flat spacetime to make it curved? Is a flat spacetime with strings ("extended tensor structures") imposed on it a curved spacetime?
In contrast, Loop Quantum Gravity comes closer to quantizing spacetime itself, there is no background spacetime through which the gravitons travel. Spacetime is already quantized from the start. The theory is background independent. Gravitons are small spacetime distortions (so not closed strings in flat spacetime). But how are elementary particles represented, if not point-like or string-like? Is there some common interface with string theory?
So, is general relativity compatible with quantum gravity? Personally I think it is. That is if you quantize spacetime itself, to which LQG comes closest. String theory is not compatible with general relativity, even if some low energy approximation reproduces the curved spacetime of general relativity. This is simply because the theory describes the gravitational interaction as taking place in a flat spacetime.
Questions, questions, questions... Meant as an answer to the difficulty in quantizing gravity. Non-renormalizability, said to be a major obstacle to quantizing gravity, seems to be of less interest, though finding a renormalizable theory can help in understanding the quantizing of spacetime itself.