First and second order phase transitions Recently I've been puzzling over the definitions of first and second order phase transitions. The Wikipedia article starts by explaining that Ehrenfest's original definition was that a first-order transition exhibits a discontinuity in the first derivative of the free energy with respect to some thermodynamic parameter, whereas a second-order transition has a discontinuity in the second derivative.
However, it then says 

Though useful, Ehrenfest's classification has been found to be an inaccurate method of classifying phase transitions, for it does not take into account the case where a derivative of free energy diverges (which is only possible in the thermodynamic limit).

After this it lists various characteristics of second-order transitions (in terms of correlation lengths etc.), but it doesn't say how or whether Ehrenfest's definition can be modified to  properly characterise them. Other online resources seem to be similar, tending to list examples rather than starting with a definition.
Below is my guess about what the modern classification must look like in terms of derivatives of the free energy. Firstly I'd like to know if it's correct. If it is, I have a few questions about it. Finally, I'd like to know where I can read more about this, i.e. I'm looking for a text that focuses on the underlying theory, rather than specific examples.
Modern Classification
The Boltzmann distribution is given by $p_i = \frac{1}{Z}e^{-
\beta E_i}$, where $p_i$ is the probability of the system being in state $i$, $E_i$ is the energy associated the $i$-th state, $\beta=1/k_B T$ is the inverse temperature, and the normalising factor $Z$ is known as the partition function.
Some important parameters of this probability distribution are the expected energy, $\sum_i p_i E_i$, which I'll denote $E$, and the "dimensionless free energy" or "free entropy", $\log Z$, where $Z$ is the partition function. These may be considered functions of $\beta$. 
It can be shown that $E = -\frac{d \log Z}{d \beta}$. The second derivative $\frac{d^2 \log Z}{d \beta^2}$ is equal to the variance of $E_i$, and may be thought of as a kind of dimensionless heat capacity. (The actual heat capacity is $\beta^2 \frac{d^2 \log Z}{d \beta^2}$.) We also have that the entropy $S=H(\{p_i\}) = \log Z + \beta E$, although I won't make use of this below.

A first-order phase transition has a discontinuity in the first derivative of $\log Z$ with respect to $\beta$: 

Since the energy is related to the slope of this curve ($E = -d \log Z / d\beta$), this leads directly to the classic plot of energy against (inverse) temperature, showing a discontinuity where the vertical line segment is the latent heat:

If we tried to plot the second derivative $\frac{d^2 \log Z}{d\beta^2}$, we would find that it's infinite at the transition temperature but finite everywhere else. With the interpretation of the second derivative in terms of heat capacity, this is again familiar from classical thermodynamics.

So far so uncontroversial. The part I'm less sure about is how these plots change in a second-order transition. My guess is that the energy versus $\beta$ plot now looks like this, where the blue dot represents a single point at which the slope of the curve is infinite:

The negative slope of this curve must then look like this, which makes sense of the comment on Wikipedia about a [higher] derivative of the free energy "diverging".

If this is what second order transitions are like then it would make quite a bit of sense out of the things I've read. In particular it makes it intuitively clear why there would be critical opalescence (apparently a second-order phenomenon) around the critical point of a liquid-gas transition, but not at other points along the phase boundary. This is because second-order transitions seem to be "doubly critical", in that they seem to be in some sense the limit of a first-order transition as the latent heat goes to zero.
However, I've never seen it explained that way, and I have also never seen the third of the above plots presented anywhere, so I would like to know if this is correct.
Further Questions
If it is correct then my next question is why are critical phenomena (diverging correlation lengths etc.) associated only with this type of transition? I realise this is a pretty big question, but none of the resources I've found address it at all, so I'd be very grateful for any insight anyone has.
I'm also not quite sure how other concepts such as symmetry breaking and the order parameter fit into this picture. I do understand those terms, but I just don't have a clear idea of how they relate to the story outlined above.
I'd also like to know if these are the only types of phase transition that can exist. Are there second-order transitions of the type that Ehrenfest conceived, where the second derivative of $\log Z$ is discontinuous rather than divergent, for example? What about discontinuities and divergences in other thermodynamic quantities and their derivatives?
 A: I'll give a very qualitative answer / overview. 
The classification 'first-order phase transition vs. second-order phase transition' is an old one, now replaced by the classification 'first-order phase transition vs. continuous phase transition'. The difference is that the latter includes divergences in 2nd derivatives of $F$ and above - so to answer your question, yes there are other orders of phase transitions, in general.
Note that there are phase transitions that do not fall into the above framework - for example, there are quantum phase transitions, where the source of the phase transitions is not thermal fluctuations but rather quantum fluctuations. And then there are topological phase transitions  such as the Kosterlitz–Thouless transition in the XY model.
The framework to understanding the thermal phase transitions is statistical field theory. A very important starting point is Ginzburg theory, and then you upgrade it to Landau-Ginzburg theory. In a nutshell, phases are distinguished by the symmetries they possess. For example, the liquid phase of water is rotationally symmetric and translationally symmetric, but the solid phase (ice) breaks that rotational symmetry because now it only has discrete translational symmetry. So there must be some phase transition between these two phases. Liquid and gas possess the same symmetry and so actually can be identified as the same phase, as evidenced by being able to go from liquid to gas by going around the critical point instead of through the liquid-gas boundary in the phase diagram. LG theory involves writing a statistical field theory of the system respecting symmetries of the system, and then studying how the solution to the field equations respects the symmetry or not against the temperature. 
Now we don't really deal with first order phase transitions as much as continuous phase transitions. I can give a few reasons:


*

*First-order phase transitions aren't very interesting. You can model them by Landau-Ginzburg theory in the mean field approach by adding appropriate terms in the effective action (like $m^3$, $m^4, m^5, m^6$, $m$ being the order parameter [yes, note that odd terms are allowed - they explicitly break the symmetry. Although for reasons of positive-definiteness the largest power must be even.]). 

*First-order phase transitions depend on the microscopic details of the system, so we don't learn much information about such a PT from analyzing one system.

*Or perhaps, we just don't know how to really deal with first-order phase transitions very well. 

*Continuous phase transitions have a diverging correlation length (first order ones typically do not). This implies a few very important things:


*

*a) Microscopic details are washed out because of the diverging correlation length. So we expect continuous phase transitions to be classified into universality classes. By that I mean that near such a critical point, thermodynamic properties diverge with some critical exponents with the order parameter, and these set of critical exponents fall into classes that can be used to classify different PTs. Refer to Peskin and Schroeder pg 450 - we see that the critical point in a binary liquid system has the same set of exponents as that of the $\beta$-brass critical point! And the critical point in the EuO system is the same as the critical point in the Ni system. Interesting, no?

*b)  We can use established techniques such as renormalization to extract information of the critical exponents of the critical points. Try this paper by Kadanoff.
Ok, so as I said this is a very qualitative answer, but I hope it points you in some (hopefully right) direction. 
A: I'll give an alternate view of how second order phase transitions may look like. Let us study a parameter $F$. If there is a second order phase transition in $F$, then the second derivative would be discontinuous but won't diverge, as for the heat capacity below:

and first derivative should look like:

Interestingly, the plot of $F$ itself may be continuous and differentiable everywhere. The first derivative is non-differentiable at a point because the functional form of $F$ is changing, although we can't see it in the plot.

It is non differentiable at the special point because the function $F$ is actually a piecewise function as below:
$$F= \begin{cases} 
      \frac{3x^2}{2} & 0\leq x\leq 3 \\
      13.5+2(x-3)^2 & 3\leq x 
   \end{cases}
$$
A: I believe there are "second-order transitions of the type that Ehrenfest conceived" in superconductors (http://en.wikipedia.org/wiki/Superconductivity#Superconducting_phase_transition ).
One of the modern classifications of phase transitions: "first-order" and "continuous" (http://en.wikipedia.org/wiki/Phase_transition#Modern_classifications )
