What is the physical significance of $i=\sqrt{-1}$? What is the physical significance of $i=\sqrt{-1}$? 
 A: The fact that :
$$e^{ix}=cos\theta+i\,sin\theta$$
is a rather important relationship which means we can handle wave-like properties using a complex exponential if we want.
The fact that :
$$z = x+iy$$
where $z$ is a complex number and $x$ and $y$ are real values, leads to a way of describing complex numbers as being on a 2-D plane (the $x-y$ plane).  So complex numbers can be thought of as representing two values which transform in a particular ways (to match the complex arithmetic).
Combining these two things means we can sometimes model physical systems using complex numbers.
However, there's no intrinsic physical reality associated with complex numbers.  They're simply an extension of real numbers and form a more complete system of mathematics.  But there are more sophisticated forms of numbers (like quaternions) which go beyond even complex numbers and physicists have found uses for those too.  We tend to find uses for things like that - well "we" being people smarter than me usually.
If complex numbers can said to have any physical significance it's that if you end up with results giving you complex values for things like distance, mass and energy (which ought to be real numbers), then the result is impossible and either the scenario you're modeling is impossible or you messed up the maths somewhere.
A: The symbol $i$ is just that: a symbol. It is a mark on a piece of paper. It has just as much physical significance as $1$. We humans invent and reinvent rules for manipulating these symbols so that, at the end of a computation, the marks at the bottom of the page correspond with some aspect of reality (hopefully).
This symbol has different meanings in different contexts, just like $1$. The rules for working with $i$ make it amenable to describing periodic phenomena: AC circuits, waves, and rotations, to name just a few. Fourier transforms are a frequent and powerful way to use complex numbers in this area.
Unless you're a mathematical Platonist, numbers don't have physical significance. They are just very rigorous adjectives that we can use to describe our world.
A: Rotations in $2$-dimensional space are equivalent to multiplication by unit complex numbers, because $SO(2)$ is isomorphic to $U(1)$. As a result, $e^{ix}=\cos x + i\sin x$ is periodic. In classical physics, this makes some planar-orbit and wave-equation problems easier to solve. But it was not until quantum theory that a deeper physical relevance of complex numbers emerged. I discuss that in the next paragraph (a fuller treatment is given here).
The conservation of probability requires that infinitesimal changes in a quantum system be governed by complex-exponential operators (which are unitary) instead of their real-exponential counterparts. Factors of $i$ emerge all over the place as a result, such as in the energy and momentum operators (resulting in the Schrödinger equation, a heat equation with an $i$ factor) and in commutators of observables (the observables are Hermitian, so their commutators are anti-Hermitian i.e. $i$ times a Hermitian quantity; a real theory cannot similarly scale symmetric operators to obtain commutators of symmetric operators, which are the real counterpart of Hermitian ones).
