Does Wick rotation work for quantum gravity? The Euclidean Einstein-Hilbert action isn't bounded from below.
Good answer from Lubos, as always. My take is slightly different, perhaps less optimistic:
For ordinary quantum field theory in flat spacetime, Wick rotation is a great calculational tool for many things you might be interested in: vacuum correlation functions of time ordered products of local operators, or static quantities in thermal equilibrium. One does Wick rotation, and then you can think about the path integral literally as an integral, albeit infinite dimensional.
But, even in this simpler context, try to calculate more complicated quantities and you'd find that Euclidean prescription either doesn't exist or is not all that useful. Examples are: time dependent quantities in thermal equilibrium, observables in non-equilibrium situations, retarded or advanced correlation functions etc. etc. For many of these calculations there is a prescription of complexifying time, and specifying an appropriate contour in complex time (or frequency) plane. This is subtle business, and even when this works it is clearly not just "rotation" to Euclidean time.
Seems to me the strongest argument to thinking about Euclidean formulation as being more fundamental, even though it doesn't cover all possible situations that Lorentzian physics presents us with, is the desire to understand the path integral literally as an integral, or at least an analytic continuation of something which can be defined as a convergent integral.
Perhaps this is somewhat well-motivated in QFT, but it is probably not the case in quantum gravity, away from the semi-classical limit. There is no sense in which you integrate over metrics to define quantum gravity. The Einstein-Hilbert action is not renormalizable so you cannot think about the path integral over metrics as giving you any information beyond leading semi-classical (saddle point) approximation. I think for these, and many more elaborate reasons, many people I know are suspicious of any claims made using Euclidean quantum gravity beyond that leading order.
As for the specific question about the conformal factor: it has the wrong sign, so the Euclidean path integral does not converge and you have to define it by further analytic continuation of the conformal factor. As always with analytic continuation, this can have some subtlties, and when we are presented with choices of prescription we are less able to be guided by physics to understand the meaning of the various possibilities.
The Wick rotation is surely a natural method to apply even in the gravitational case. And by the way, it's very likely that the Euclidean calculation is always more fundamental and well-behaved than the Minkowski one.
As you correctly say, the Einstein-Hilbert action isn't bounded from below (and above) in the Euclidean space. But it is of course unbounded in the Minkowski spacetime. So by switching to the Euclidean spacetime, you surely don't make things worse.
Even more importantly, the unboundedness isn't a sign of a problem with a particular method: it is an actual physical insight. The action can go negative because of the "conformal mode" of the metric tensor. If you visualize the metric tensor as a collection of scalars, the "conformal mode" - one linked to the determinant of the metric - is the single scalar field with the wrong sign of the kinetic term.
This difference of the sign arguably has cosmological consequences. It's also the reason why moduli spaces of gravitational theories are covered by "well-understood" patches in 2+1 dimensions or higher but they produce new, unknown regions in 1+1 dimensions or lower:
The Euclidean spacetime representation is arguably even more important in quantum gravity than it is in non-gravitational quantum field theories. In particular, the Euclidean continuation of the black hole solutions make the right value of the temperature transparent: the temperature is such that the asymptotic periodicity of the solution at infinity is $\beta$ (in the Euclidean spacetime), the interior of the black hole is totally removed from the solution, and the event horizon is a smooth places of the solution without any deficit (or excess) angle.
Because general relativity allows many topologies and many ways to choose the time coordinate globally or locally, it allows many ways to Wick-rotate the spacetime. This diversity recently began to be used by brave physicists. I mean e.g. this paper by Ashoke Sen:
Just to be sure, the Hartle-Hawking wave function is another key paradigm in the Euclideanized quantized general relativity. It postulates that the initial conditions near the Big Bang are such that the Euclidean spacetime is smooth around the origin.
Wick rotation might work for backgrounds with an asymptotically timelike Killing vector isometry like asymptotically flat metrics and asymptotically anti de Sitter. It might also work when there's an asymptotic Killing vector which is either timelike to null at some horizon with what lying beyond truncated. Examples are stationary black holes and de Sitter space. On the other hand, Wick rotation can't possibly apply to backgrounds like Friedmann-Robertson-walker. If anyone disagrees with me, please feel free to point out why.
I am a firm believer of notion of wick rotation. As Lubos nicely pointed out, the most clear indication of wick rotation comes from quantum cosmology. In fact, many new recent results in different models of quantum gravity such as loop quantum gravity$^1$ (LQG), causal dynamical triangulation$^2$ (CDT), non-commutative geometry$^3$, Horava-Lifshitz Gravity$^4$, causal sets$^5$, and eventually in superstring M-theory$^6$, strongly indicate that at the very high curvatures near or at the Planck scale (totally non-perturbative quantum gravitational system) the signature of spacetime is not Lorentzian anymore and turns into a quantum version of 4-D Euclidean space which is timeless, acausal and devoid of any dynamics or evolution, describing elegantly the so called No-Bondary Proposal of Hartle-Hawking.
$^1$See M.Bojowald, J.Mielczarek, A.Barrau, J.Grain, G.Calcagni
$^2$See Ambjorn, Coumbe, Gizbert-Studnicki, and Jurkieiwcz
$^3$See Lizzi, Kurkiv, D'Andrea, and others
$^4$See Horava, Melby-Thomson, and Kevin Crosvenor, as well as the K. Crosvenor lecture at the GR21 conference 2016
$^5$See L. Glaser and S.Surya, as well as Glaser, D.O'Connor and Surya; forthcoming
$^6$See C.Hull, Edward Witten, Robbert Dijkgraaf, Ben Heidenreich, Patrick Jefferson, and Cumrun Vafa 2016