Phase Transitions from a Bayesian Perspective of Statistical Mechanics I have been reading papers by E.T. Jaynes recently about viewing all of statistical mechanics as just Bayesian inference applied to physics. (For an introduction: https://journals.aps.org/pr/abstract/10.1103/PhysRev.106.620) 
I find this perspective very elegant and enlightening, especially compared to the conventional point of view when being introduced to statistical physics, which I believe creates a great deal of unnecessary confusion by conflating epistemology and ontology. It also has the added advantage that it doesn't rely on the ergodic hypothesis, or require things to reach "thermal equilibrium", since temperature is just a Lagrange multiplier for making a maximally unbiased probability distribution that accounts for an expected energy. When the system is too complicated to consider any further relevant quantities or effects, we ignore them, much like we often do when estimating the probabilities of "random" event's like a coin being tossed in regards to the initial conditions of it's orientation, velocity, and angular momentum. If we notice effects or quantities that are "non-ergodic" or that our system is "non-equilibrium", we can simply add in the additional relevant information with more Lagrange multipliers. 
However, there is a situation in which I'm having a hard time wrapping my head around the Bayesian statistical physics point of view. I want to hear an explanation of phase transitions that explicitly acknowledge temperature as an epistemic quantity, and not ontological.
The Bayesian perspective seems to imply that different phases themselves are epistemic, and that we just categorize collections of micro-states as being in different phases. For example, it seems nonsensical to consider whether a single molecule of H2O is either in a "liquid" or "gas" state, so what are we really saying when $~10^{20}$ are? Is there a particular length scale or number of molecules I can start calling it a "liquid" or "gas" (Normally statistical physics considers the infinite limit)? Are these just arbitrary empirical categorizations? 
There is another example where I believe thinking in terms of temperature obscures the physical ontology. The Hamiltonian of a ferromagnet has an $SO(3)$ rotational symmetry, but we are taught that below a critical temperature the spins all settle into alignment and the $SO(3)$ breaks into an $SO(2)$ symmetry.  
However, if we consider a single spin while the ensemble is above the critical temperature, we find it is already breaking the $SO(3)$ symmetry of the Hamiltonian since the spin itself is pointing in a particular direction. Additionally if we have domain walls after the spins align below the critical temperature, then we still have an $SO(3)$ symmetry when considering length scales larger than the size of the domain walls. It appears that the "phase" depends entirely on the length scale considered, so it's not clear what role temperature plays in this.
I suppose my question can be abbreviated to this:
What is happening during symmetry breaking or a phase transition from an epistemic, Bayesian perspective?   
Edit: Grammar
 A: First, a macrostate in statistical mechanics does not correspond to a particular realization of the system (say, spins), but to a probability measure on the set of microscopic configurations (or microstates). So, when one says that the system is invariant under the action of a particular symmetry group, this means that the measure is invariant, not the particular realization. In particular, this holds even for a single spin in your system, since it is equally likely to point in each direction (and so its distribution is indeed isotropic).
In some situations, there may be several distinct macrostates for the same set of thermodynamical parameters. In such a case, one says that there is a first-order phase transition, and each of these probability measures (or rather, each of the extremal ones) corresponds to a phase of the system.
Concerning your second point, about domain walls. In simple systems (say, the nearest-neighbor classical ferromagnetic Heisenberg model, or the Ising model), there will be no domain walls at equilibrium (there will be small, localized excitations, but the system does not split into large regions with different orientations of the spins). If you consider more complicated systems in which such domain walls do appear and are stable, then I would not say that they undergo a symmetry breaking.
Finally, I wouldn't say that the temperature in Jaynes' approach is epistemic. For a large system, it is an unambiguously defined quantity that possesses all the properties ascribed to the thermodynamic temperature and, as such, possesses an objective meaning. What is nice in this approach is that you start with an a priori ambiguous, subjective point of view (describing your knowledge of the system rather than its actual state), but you end up with completely deterministic (and thus objective) predictions (for very large systems). In a sense, in macroscopic systems, the measure maximizing your ignorance already provides complete, deterministic information on macroscopic properties.
