I am a graduate student in mathematics working in probability (without a very good background in physics honestly) and I've started to see arguments based on computations derived from the replica trick. I understand that it is non-rigorous but it appears that a decent number of the solutions one obtains using this trick can be made rigorous by other (often fairly involved) methods. I would like to work on developing a better heuristic approach to problems, so I would like to understand when I can reasonably expect this type of argument to give reasonably accurate predictions. Perhaps more importantly for me, is there a nice characterization of physical situations when it is clear that this trick should fail?
'Critique of the replica trick' written in 1985 by Verbaarschot and Zirnbauer seems to be a good starting point to answer the last part of your question
Perhaps more importantly for me, is there a nice characterization of physical situations when it is clear that this trick should fail?
In their introduction, they explain that soon after its introduction by Edwards and Anderson in 1975, an example was found by Sherrington and Kirkpatrick where the replica trick gave unphysical results (a negative entropy at low temperatures for an Ising spin glass with infinite range interactions).
Then they refer to a mathematical argument made by van Hemmen and Palmer in 1979, who attribute the problem to the non-uniqueness of the analytic continuation $n \rightarrow 0$ (rather than an order of limits issue as speculated by Sherrington and Kirkpatrick).
Even if they agree with the mathematical content of the paper by van Hemmen and Palmer, their own paper seeks to give more insight into the physical mechanism that causes the replica trick to break down.
Their conclusion is that problems with the replica trick may occur whenever the theory for a general integer value of $n$ doesn't have the same symmetries as the theory for $n = 0$.
As far as I know, the failure of the replica trick and its convoluted connections with the aforementioned symmetry breaking, the non-uniqueness of the analytic continuation, and order of limits issues, is still a hot topic, as you can find it in some recent avatar of the same problem, see e.g. the discussion at the top of page 27 of 1901.04499.