Determinant of d'Alembert Operator $\mathop\Box-m^{2}$ In quantum field theory, the partition function of a free scalar is 
$$\mathcal{Z}=\int\mathcal{D}\phi\exp i\int d^{n}x\frac{1}{2}\left[(\partial_{\mu}\phi)(\partial^{\mu}\phi)-m^{2}\phi^{2}\right]$$
$$=\mathrm{Det}^{-1/2}(\mathop\Box-m^{2}).$$
My question is on what spacetime manifold does the d'Alembert operator $\mathop\Box-m^{2}$ have zero modes. If it has zero modes, how does one define the above path-integral?
 A: Well for example Minkowski spacetime (just plug in a plane-wave ansatz).
The malaise arrives because the formula with the determinant is valid for Gaussian integrals, i.e.
$\int e^{- \pi A_{ij} x^i x^j} d x = \frac{1}{\sqrt{\det A}} \ . $
Here $A$ is symmetric positive definite. First i will show that the same holds if in the exponent there is an $i$, i.e.
$\int e^{i\pi A_{ij} x^i x^j} d x = \frac{1}{\sqrt{\det A}} \ . $
To do so, deform the integration contours as to obtain an integration of $e^{i \pi A_{ij} x^i x^j}$. For simplicity consider the one-dimensional case only, i.e. deform the integration contour of 
$\int_{-\infty}^{\infty} e^{-\pi a x^2} d x =  \frac{1}{\sqrt{a}} \ .$
Approximate the integrals by an integral over the interval $[-R,R]$. Now decompose this contour in a contour $\gamma_1$ from $x = -R$ to $x = -(R + i R)$, then a contour $\gamma_2$ from $ x = - (R-iR)$ to $x = R+ i R$ and then a contour $\gamma_3$ from $x = R-iR$ to $x = R$. We have:
$  \int_{-R}^{R} (...) d x = \int_{\gamma_1}(..)dz + \int_{\gamma_2}(..)dz + \int_{\gamma_3}(..)dz \ ;$
or, more explicitely:
$\int_{-R}^{R} e^{-\pi a x^2} d x = \int_{-\frac{R}{\sqrt{2}}}^{\frac{R}{\sqrt{2}}} e^{i \pi a x^2} d x  + I(\pi a R^2) \ , \  I(w) = 2 e^{-w} \int_0^1 e^{w x} \cos(2 x w) d x \ .$
If we can argue that $I(w) \rightarrow 0$ for $w \rightarrow \infty$, then we have obtained our formula. This however is not so hard, first note that
$I(w) \leq 2 e^{-w} \int_0^1 e^{w x}  d x $
and then use the first mean value theorem for definite integrals (cf. https://en.wikipedia.org/wiki/Mean_value_theorem#Mean_value_theorems_for_definite_integrals) to conclude
$I(w) \leq 2 e^{-w(1-x_0^2)} \ , $
with $x_0$ strictly smaller than 1. This is enough to conclude our formula and thus
$\int_{-\infty}^{\infty} e^{-\pi a x^2} d x =\int_{-\infty}^{\infty} e^{i \pi a x^2} d x$
Now comes the punchline. The above argument shows that for a positive definite $A$ one may freely switch between having an $i$ in the exponent or a $-1$. However the operator in your question, $\Box - m^2$, has positive, negative and zero eigenvalues, where the Gaussian integral no longer makes sense, while the imaginary Gaussian still might make sense.
In fact, consider the same argument with contours from $x=-R$ to $x = -(R+iR)$ etc., which gives
$\int_{-\infty}^{\infty} e^{-\pi a x^2} d x =\int_{-\infty}^{\infty} e^{- i \pi a x^2} d x$
Hence reading this backwards leads to the conclusion
$\int e^{i\pi A_{ij} x^i x^j} d x = \frac{1}{\sqrt{|\det A}|} \ . $
Hence the negative eigenvalues do not pose a problem. What about the zero eigenvalues? Well they are a problem in general but they may drop out in a similar way how one can integrate a square root singularity. 
In particular, if $A$ has an eigenvector $v$ with $Av = 0$, then the Gaussian integral has a flat direction and the determinant is infinite. However, going to infinite-dimensional vector spaces allows for more interesting things to happen. In this case the Gaussian integral may or may not make sense as a limit of finite-dimensional integrals; here i will make some remarks on the determinant.
Let's take $A$ to be a self-adjoint operator. Then the spectrum of $A$ consists of the point spectrum, i.e. real numbers $\{\lambda_i\}_{i\in \mathbb{N}}$ s.t. there are vectors $v_i$ with $A v_i = \lambda_i v_i$. But additionally, A may also have a continuous spectrum. This is the set of $\lambda$ for which $\lambda - A$ is injective, but not surjective (additionally one also asks it to have dense spectrum). Roughly speaking, this means while $\lambda - A$ maps all vectors in the vector space to non-zero vectors, it is still not possible to invert it. The reason is that there might be approximate eigenvalues, for example how narrow wavepackets are almost eigenstates of the derivative operator. Anyhow this means that if we take now the logarithm of the determinant it consists of two terms:
$ \log |\det A|^{-\frac{1}{2}} = - \frac{1}{2} \sum_{i =1}^\infty \log|\lambda_i| - \frac{1}{2} \int_0^\infty \rho(\lambda) \log \lambda d \lambda  \ .$
Here $\rho(\lambda)$ is a positive function, indicating how many eigenvalues are in an interval $[\lambda,\lambda + d\lambda]$.
Note that if $A$ is a differential operator, one typically has to regularize this expressions, but this will not be of interest here. Instead, let's focus how zero eigenvalues can become a problem. First of all, like in the finite-dimensional cases, if a proper eigenvalue is zero, the determinant vanishes. But what if the continuous spectrum extends down to zero? we can see from the formula, this is not too problematic, since $\log(\lambda)$ is integrable at 0, as long as $\rho(\lambda)$ is not too singular for small $\lambda$.
Let's do an example, the real scalar field on Minkowski spacetime. Then $A = \Box - m^2$ has only continuous spectrum:
$ \log Z = -\frac{1}{2} \int \frac{d^n p}{(2\pi)^n} \log|p_0^2 - \vec{p}^2 - m^2| =  -\frac{1}{2} \int_0^\infty \log(\lambda) \rho(\lambda) d \lambda $
with the function
$\rho(\lambda) = \int \frac{d^n p}{(2\pi)^n} \delta(\lambda - |p^2 - m^2|) = \rho_< + \rho_> \  $
with
$ \rho_<(\lambda) = 2 \int \frac{d^{n-1} p}{(2\pi)^n} \int_{0}^{\sqrt{\vec{p}^2 +m^2}}\delta(\lambda + p_0^2 - \vec{p}^2 - m^2) d p_0  = \\ =  \frac{\text{Vol}(S^{n-2})}{(2\pi)^n} \int_0^\infty p^{n-2} \frac{\theta(p^2 + m^2 - \lambda)}{\sqrt{p^2 + m^2 - \lambda}} dp  $ 
and
$ \rho_>(\lambda) = 2 \int \frac{d^{n-1} p}{(2\pi)^n} \int_{\sqrt{\vec{p}^2 +m^2}}^\infty \delta(\lambda - p_0^2 + \vec{p}^2 + m^2) d p_0  = \\ =  \frac{\text{Vol}(S^{n-2})}{(2\pi)^n} \int_0^\infty p^{n-2} \frac{1}{\sqrt{p^2 + m^2 + \lambda}} dp  $ 
subtracting the infinite constant $\rho(0)$ we get the renormalized density
$\rho(\lambda) -\rho(0) = \frac{\text{Vol}(S^{n-2})}{(2\pi)^n} \int_0^\infty p^{n-2} \left[\frac{1}{\sqrt{p^2 + m^2 + \lambda}} + \frac{\theta(p^2 + m^2 - \lambda)}{\sqrt{p^2 + m^2 - \lambda}} - \frac{2}{\sqrt{p^2 + m^2}} \right] dp $
Let's check what is the behaviour for small $\lambda$. For simplicity specialize to $n = 4$.
$\rho(\lambda) -\rho(0) \sim \frac{3 \lambda^2}{16\pi^3} \int_0^\infty \frac{p^2}{(p^2 + m^2)^{\frac{5}{2}}} d p \sim \frac{1}{16\pi^3} \frac{\lambda^2}{m^2} \ \ \ \ \text{ as} \ \ \lambda \rightarrow 0 \ . $
Hence there seem to be no problems from zero eigenvalues here.
