My understanding is that within Landau theory, a continuous phase transition is only possible if certain symmetry rules are satisfied. (These rules represent necessary but not sufficient conditions for a continuous phase transition.) One criterion is that the symmetry group of one phase must be a subgroup of the other phase's symmetry group (cf. p. 782 of Rep. Prog. Phys. 50, 783). What are the other criteria, and where can I find a derivation of them?
This "criteria" is neither sufficient nor necessary.
There are many symmetry-breaking transitions which are not continuous. For example, it is well-known that $n$-state Potts model has a thermal symmetry-breaking phase transition, and when $n>4$ it is first order. For a more realistic one, I think melting transition is first order. Basically there is no way to tell whether a transition is continuous or not just from the symmetry. It really depends on the details a lot.
There are also "exotic" continuous phase transitions between states with the same symmetry, or states with very different symmetry breaking. A famous example of the latter type is the so-called deconfined quantum criticality, which happens in lattice spin models where one phase spontaneously breaks translation symmetry and the other phase spontaneously breaks spin rotation symmetry. Neither is a subgroup of the other. The critical theory is a gauge theory, going beyond the scheme of Landau theory.