Spontaneous Symmetry Breaking - struggling with physics based understanding?

Although I am a mathematician by nature, I'm writing an essay in my third year of my undergraduate on Spontaneous Symmetry Breaking in Physics, and as such I've become a little confused by how the logic works in showing that the symmetry of a system is spontaneously broken. Anyway, I'm quite confused on the general process. For example, in the simplest instance of symmetry breaking (a breaking of $\mathbb{Z}_2$ symmetry), we have a Lagrangian of the form: $$\mathfrak{L} = \dfrac{1}{2}(\partial_{\mu}\varphi)^2 - V(\varphi)$$ Where $V(\varphi) := -\dfrac{m^2}{2}\varphi^2 + \dfrac{\lambda}{4}\varphi^4$ and the Lagrangian clearly obeys $\mathbb{Z}_2$ symmetry. Here, the corresponding energy is given by: $$E = \int d^3x \bigg(\dfrac{1}{2}(\partial_{0}\varphi)^2 + \dfrac{1}{2}(\partial_{i}\varphi)^2 - \dfrac{m^2}{2}\varphi^2 + \dfrac{\lambda}{4}\varphi^4\bigg)$$

Now the first question I have is, why do we now consider the ground-state in the first place? What is the reason for this? The second question I have is, once we have the ground states, why do we consider perturbations around a chosen ground state? I understand an argument I've found in one source as to why we may choose any ground state on an equal footing but my main gripe is with why on earth we are considering perturbations in the first place.

I hope this makes some sense, and sorry if it's an inane question - my understanding of Physics isn't perfect!

The reason behind the fact that we build our theories around perturbations of a ground state is simply that solving these equations exactly is not feasible. Hence, we try having perturbative solutions that are approximations based on supposing that the interactions are small enough that they don't deform too much the solutions to the case when there are no interactions (free case) which can be solved. Since we have assumed the interactions are small, generally this implies that the energy is relatively speaking small, so we are seeking solutions that don't depart too much form the case where the energy is the smallest possible, that is, a ground state. That is the main intuitive reason why the theories are built around a ground state.

Now, with spontaneous symmetry breaking, the ground state is degenerate and we have several choices. Hence, small perturbations around one of these ground states do not reflect the symmetry of the full theory at low energy because we are tied to perturbations around one of these ground states. Now the question when studying a real world system is which ground state will we use? They are all equal in the theory but we are physically "near" one of them.

The consideration of ground states is an equilibrium assumption.

Imagine that instead of developing a theory around a ground state we were doing it around any general state. Actually, for simplicity I'll only consider states with constant $\varphi$, but the same remarks will be valid with the appropriate modifications -- modulo some topological considerations that can be interesting but lead to a rabbit hole squarely outside the scope of this answer (key words: solitons, instantons, monopoles, kinks, domain walls...).

Anyhow, making the transformation $\varphi \rightarrow \varphi - a$, the potential will be

\begin{align} V(\varphi) &= -\frac{m^2}{2}(\varphi-a)^2 + \frac{\lambda}{4}(\varphi-a)^4 \\ &= \left(\frac{a^4 \lambda}{4}-\frac{a^2 m^2}{2}\right) + \bigg(a m^2 - a^3 \lambda\bigg)\varphi + \left(\frac{3}{2} a^2 \lambda-\frac{m^2 }{2}\right)\varphi^2 - a\lambda\varphi^3 +\frac{\lambda}{4}\varphi^4 \end{align}

Let's imagine that $\varphi$ is very small. This does correspond to considering a perturbation around this chosen state but in this context you can just view this as a stability analysis: I want to see how the theory responds if the field happens to deviate a little from the choice I made. This is similar to figuring out if a pencil balanced on its tip can remain balanced by seeing what happens when I give it a little push.

Then the potential becomes

$$V(\varphi) \approx \bigg(a m^2 - a^3 \lambda\bigg)\varphi + \mathcal{O}(\varphi^2)+ \text{irrelevant constants}$$

(for clarity, I'll omit the constants and higher order terms from now on)

If the coefficient in front of $\varphi$ is positive, you can lower the energy by decreasing $\varphi$. If it's negative, you can lower the energy by increasing $\varphi$. It follows that the only way the theory can possibly be stable is if the coefficient is exactly zero.

This rules out an equilibrium theory about any state other than extrema of the extrema of the potential (more generally we might also want to consider extrema of the action but once more we're led outside the scope of this answer).

Then let us say that we're considering the theory about points $a$ for which the leading term vanishes. There are three such points: $a = 0$, and $a = \pm \sqrt{\frac{m^2}{\lambda}}$. Unsurprisingly they are the extrema of the potential as I anticipated above. The potential for small $\varphi$ will be

$$V(\varphi) \approx \left(\frac{3}{2} a^2 \lambda-\frac{m^2 }{2}\right)\varphi^2$$

For $a$ = 0 we have

$$V(\varphi) \approx -\frac{m^2 }{2}\varphi^2$$

And for $a = \pm\sqrt{\frac{m^2}{\lambda}}$ we have

$$V(\varphi) \approx \left(\frac{3}{2} m^2 -\frac{m^2 }{2}\right)\varphi^2 = m^2\varphi^2$$

We have been irrevocably led to the standard story: since in the case where $a = 0$ you may decrease the energy by increasing $\varphi$, the theory won't be stable with respect to small perturbations, so the only possible stable states around which to develop the theory are the two ground states.

I guess it will be helpful if we draw an analogy to equilibrium thermodynamics. An isolated system in thermodynamic equilibrium will typically be sitting on a minimum of the free energy

$$F = U - TS$$

This isn't to say that other states aren't possible or important, just that they're not at equilibrium. As a consequence, they're much harder to analyze. We're fortunate that the universe seems to be more or less in an equilibrium state so that the simplifying assumption of stability is actually experimentally relevant. At least as far as fundamental field theories are concerned, anyway: for instance, we don't have to consider corrections to the mass of the electron due to the expansion of the universe.