For me a phase transition is defined (somewhat strictly) as a non-analyticity in the thermodynamic potential. A function $f(x)$ is non-analytic at $x=x_0$ if it can not be expanded in powers of $x-x_0$. For example $f(x) = \sqrt{x+1}$ can be expanded around $x=0$
$$f(x) \cong 1 + \frac{x}{2} - \frac{x^2}{8} + \dots $$
but this does not work at $x=-1$ because the first derivative of $f(x)$ diverges at $x=-1$. A more realistic example would be a function like $f(x) = \sqrt{x^2+m^2}$ with $m$ to be interpreted as some parameter of the system. We see that $f(x)$ is analytic for all values of $x$, except when $m=0$ and then we get a cusp $f(x) = \left|x\right|$. This means that there is a special point in the phase diagram where the free energy is non-analytic. This is a phase transition.
For example, in the Ising model, the free energy is a continuous function of the magnetic field $h$, but is has a cusp at $h=0$. Then the magnetisation (which is the derivative of the free energy with respect to the magnetic field) is discontinuous. At a the critical point, the cusp goes away but the magnetisation behaves as a power law (with a non-integer exponent). A different non-analyticity emerges.
I like this definition because it is directly tied to the fact that something special happens at phase transitions. Indeed, in statistical physics the Free energy is computed by taking the logarithm of the partition function
$$ F = -k_{\text{B}} T \, \text{Ln}\left(\sum_{\text{states}} \text{e}^{-E_{\text{state}}/(k_{\text{B}T})}\right) \, .$$
This is a sum of finite elements. Then it is analytic almost by construction. The way out is that the system may contain infinitely many particles so that the sum has infinitely many elements. Think for example of the geometric series, $\sum_{n=0}^\infty x^n = 1/(1-x)$ which is not analytic at $x=1$ even though this does not appear in the individual elements of the sum. The finite sum $\sum_{n=0}^N x^n = (1-x^{N+1})/(1-x)$, is always perfectly analytic. Physically, this means that a phase transition can only occur in the thermodynamic limit. Systems without an infinity somwhere do not have phase transitions.
This is of course a bit strict and needs to be generalised some times. The idea however remains. For example
- Dynamical phase transitions can emerge when we look at the state of a system after an infinite time evolution as a function of its microscopic parameters.
- In Renormalisation Group (RG) methods, we purposefully avoid non-analiticities by introducing a cut-off, but we let the cut-off run to infinity.
- In quantum phase transitions, we send the temperature to zero (see the exponentials in the above equation where $1/T$ goes to infinity).
- In the Ginzburg-Laudau picture, the free energy is first given as an analytic function of the magnetic field and the magnetisation. The physical free energy depends however only on the magnetic field with the magnetisation being computed by looking for the minimum of the free energy. This minimisation produces the non-analyticity and is equivalent to finding the asymptotic equilibrium of a dynamical system.