For simplicity, let's consider a two dimensional version of Klein-Gorden equation: $$ (\partial_t^2-\partial_x^2-\partial_y^2+m^2) G(\vec{x},t) = -\delta(\vec{x})\delta(t) $$
From the previous posts:
How to obtain the explicit form of Green's function of the Klein-Gordon equation?
How do I Derive the Green's Function for $-\nabla^2 + m^2$ in $d$ dimensions?
both the answers suggest the analytical continuation and especially follow the answer given by @Sean Lake, we can solve this equation simply by analytical continue to a familiar equation then convert back.
attempt 1
Here is the outline of the procedure:
let $t\to iz', x\to x', y\to y'$, we have $lhs=-(\nabla'^2-m^2)G$, $rhs=-\delta(iz')\delta(x')\delta(y')=i\delta(\vec{r}')$. The equation now reads: $$ (\nabla'^2-m^2)G=-i\delta(\vec{r}') $$
The above equation is the screened Poission equation, the solution can be easily get as: $$ G=\frac{i\,e^{-mr'}}{4\pi r'} $$
Convert back using $z'\to -it, x'\to x, y'\to y$, we have: $$ G=\frac{i\, e^{-m\sqrt{x^2+y^2-t^2}}}{4\pi\sqrt{x^2+y^2-t^2}} $$ for $x^2+y^2>t^2$.
when $t^2<x^2+y^2$, we can analytically continue it to: $$ G=\frac{e^{-im\sqrt{t^2-x^2-y^2}}}{4\pi\sqrt{t^2-x^2-y^2}} $$ where we have used the $\sqrt{-1}=i$.
Two question regarding the above procedure:
question 1: since we can also change $t\to -iz'$, the lhs of the equation unchanged, while the rhs of the equation has an additional minus sign, because: $rhs=-\delta(-iz')\delta(x')\delta(y')=-i\delta(\vec{r}')$, therefore the final answer differ by an overall minus sign!
question 2: in step 4, we have used that $\sqrt{-1}=i$, but what if I use $\sqrt{-1}=-i$, it seems that it will lead to: $$ G=-\frac{e^{im\sqrt{t^2-x^2-y^2}}}{4\pi\sqrt{t^2-x^2-y^2}} $$ when $t^2>x^2+y^2$.
attempt 2
the outline:
let $t\to z', x\to ix', y\to iy'$, we have $lhs=(\nabla'^2+m^2)G$, $rhs=-\delta(z')\delta(ix')\delta(iy')=\delta(\vec{r}')$. The equation now reads: $$ (\nabla'^2+m^2)G=\delta(\vec{r}') $$
the above equation is the Helmholz equation, the solution is: $$ G=-\frac{e^{imr'}}{4\pi r'} $$
Convert back, we have $$ G=-\frac{e^{im\sqrt{t^2-x^2-y^2}}}{4\pi\sqrt{t^2-x^2-y^2}} $$ when $x^2+y^2<t^2$.
when $x^2+y^2>t^2$, we analytical continue the results, we get: $$ G=i\frac{e^{im\sqrt{x^2+y^2-t^2}}}{4\pi\sqrt{x^2+y^2-t^2}} $$ we have used $\sqrt{-1}=i$.
If I use $\sqrt{-1}=-i$, then when $x^2+y^2>t^2$, we have: $$ G=-i\frac{e^{-im\sqrt{x^2+y^2-t^2}}}{4\pi\sqrt{x^2+y^2-t^2}} $$
attempt 2 has the same problem as attemp 1. Also, does the two attemps consistent?
In summary, I am confused about the idea of analytical continuation here, which way to do and why to do so is my question. In my point of view, the above substitution can't be all correct, there must be some point I missed by the mindless substitution of variables.
In fact, I remember the solution to Helmholtz equation has two solutions, which are: $G=-\frac{e^{\pm imr'}}{4\pi r'}$, similar to Screened Poisson equation I think. This would lead to more complications (more results).