Confusion about symmetry spontaneous breaking in a simple model Hamiltonian and symmetry
I was learning about the symmetry spontaneous breaking (SSB), but confused by this simple minima model$$H=E_0\sum_{i=1}^3|i\rangle\langle i|+J\left(|1\rangle\langle 2|+|2\rangle\langle 3|+|1\rangle\langle 3|+h.c.\right)$$It's matrix form$$H=\begin{pmatrix}E_0 & J & J \\ J & E_0 & J \\ J & J & E_0\end{pmatrix}$$which describes that a particle hops on a lattice consisting of only 3 cells with on-site energy $E_0$ and hopping strength $J$, and the PBC is applied. It's clear that $H$ respects the $Z_3$ symmetry under the PBC, which means all the eigenstates of $H$, $|\psi_{i=1,2,3}\rangle$, will satisfy the symmetry condition$$Z_3|\psi_{i}\rangle=|\psi_i\rangle\ \ i\in1,2,3$$as long as the $Z_3$ symmetry was not spontaneously broken (for simplicity,
$Z_3$ denotes both the group and it's unitary representation).
Spectrum of $H$: The degeneracy of eigenvalues and the symmetry of eigenstates
The eigenvalues and eigenstates were very easy to obtain$$\left\{\begin{aligned}&E_e=E_0+2J\\&E^{(2)}_g=E_0-J\end{aligned}\right.\ \ \ \ 
\left\{\begin{aligned}&|\psi_e\rangle=\frac{1}{\sqrt{3}}(|1\rangle+|2\rangle+|3\rangle)\\&|\psi^{(2)}_g\rangle=|\psi_{a,b}\rangle\ \ \ for \ \ \langle \psi_a|\psi_e\rangle=\langle \psi_b|\psi_e\rangle=\langle \psi_a|\psi_b\rangle=0\end{aligned}\right.$$
where $E_e, E^{(2)}_g$ denotes the excited state and ground state respectively, and the superscript in $E^{(2)}_g$ denotes the two-fold degeneracy of the GS.
The unique ES $|\psi_e\rangle$ stays unchanged under the symmetry transformation $$Z_3|\psi_e\rangle=|\psi_e\rangle.$$
For the two-fold degeneracy GS, one can always choose two referance state $|\psi_a\rangle,|\psi_b\rangle$ that satisfies the orthogonality condition but changes under the action of $Z_3$, for example$$\left\{\begin{aligned}&|\psi_a\rangle=\frac{1}{\sqrt{2}}(|1\rangle-|2\rangle)\ \ \ \ \ \ \ \ \ \ \ \ \ \ Z_3|\psi_a\rangle\ne|\psi_a\rangle\\&|\psi_b\rangle=\frac{1}{\sqrt{6}}(|1\rangle+|2\rangle-2|3\rangle)\ \ \ Z_3|\psi_b\rangle\ne|\psi_b\rangle \end{aligned}\right.$$
Review some important properties of TFI model
Consider the TFI model $$H_{\mathrm{TFI}}=-\sum_{i}{h\sigma^x_i}+\sigma^z_{i}\sigma^{z}_{i+1}$$for $h<1$

1). The true GS keeps the $Z_2$ symmetry, and the two aligned states $|\mathrm{all\ up}\rangle,|\mathrm{all\ down}\rangle$ are neither degenerated nor the eigenstates of TFI model for finite $N$. 2). These two aligned states become both degenerated and the eigenstates of TFI model which breaks $Z_2$ symmetry when $N\to +\infty$.

Questions

*

*Can I say that this $H$ has SSB due to the spontaneous $Z_3$ symmetry breaking by the two-fold degeneracy GS?

*If the answer of Q1 is Yes. It's obvious that I don't need anything (like $N\to\infty$) to make $|\psi^{(2)}_g\rangle$ become degeneracy and eigen. So my question 2 is: What kinds of quantities make this symmetry breaking become "Spontaneous" in this simple model?

 A: This system won't exhibit spontaneous symmetry breaking (SSB). You need the thermodynamic limit in order to see SSB.
The point is that in an infinite system, even though the degenerate ground states are technically equivalent, to get from one to the other would require a tunneling of a macroscopic degrees of freedom which cost prohibitively large amounts of energy. In a finite system, this does not happen.
A: The question is based on misinterpretation of the meaning of terms symmetry and spontaneous symmetry breaking.
Symmetry
The Hamiltonian in the OP possesses a rotational symmetry (note that all this is easily generalizable to N sites with periodic boundary conditions). This does not mean that the eigenfunstions functions should possess the same symmetry, but only that in symmetry transformations they transform into linear combinations of the eigenfunctions, which they surely do. Where the symmetry does manifest itself is that the eigenstates correspond to the irreducible representations of the symmetry group, which they do.
Spontaneous symmetry breaking
Spontaneous symmetry breaking applies to many-body systems with symmetric Hamiltonians exhibiting non-symmetric states, such as the well-defined polarization of a ferromagnet. This is essentially a many-body/statistical-physics phenomenon, which cannot be uncovered in a system with a finite number of degrees of freedom (i.e., as @PaulMalinowski pointed in their answer, it is possible only in in thermodynamic limit).
The standard basic reading here is the article by Phil Anderson More is Different. It could be particularly relevant here, since Anderson considers an example of an ammonia molecule, which is somewhat similar to the one in the OP.
