Your general idea is correct. As an explicit example, consider a field theory of $n$ scalar fields that can be placed in an $n$-component column vector $\phi$, with a potential, $V(\phi)$. In addition, suppose a symmetry group, $G$ of the Lagrangian.
The potential $V(\phi)$ has a family of degenerate minima that form a manifold, $\mathcal M$. Now, for a theory with a vacuum expectation value, $\phi_0$, we expect $D(g)\phi_0$ for an appropriate representation, for an element $g\in G$, to be a valid minimum, if $\phi_0$ is a minimum.
However, if $G$ is spontaneously broken to a subgroup, $H$, then we expect that only $D(h)\phi_0$ for $h\in H$ to also be a valid minimum, rather than the full group, $D(g)\phi_0$.
Therefore, $D(gh)\phi_0 = D(g)\phi_0$ and we can define equivalence classes by the fact that $g_1 \sim g_2$ if and only if $g_2 = g_1 h$ for $h \in H$. Such equivalence classes you will recognise as being in the coset, $G/H$.
The vacuum manifold then is simply, $\mathcal M = G/H$. The global properties of $\mathcal M$ are particularly important in physics. For example, in three spatial dimensions, magnetic monopoles only arise if the group $\pi_2 (G/H)$ is non-trivial.
Your second question, how one would go about 'using this idea' is extremely broad; spontaneous symmetry breaking arises in nearly every branch of mathematical physics.
Application
Spontaneous symmetry breaking can be used as a mechanism to endow massless fields with mass, as is the case with the Higgs mechanism in the Standard Model.
As an example, consider an $SO(3)$ gauge theory, with three real scalars, $\phi_i$, with Lagrangian,
$$\mathcal L = -\frac14 (F^a_{\mu\nu})^2 + \frac12(\partial_\mu \phi_i - ig A^a_\mu \tau^a_{ij}\phi_j)^2 + \frac12 m^2 \phi^2_i - \frac{\lambda}{4!}(\phi_i)^4.$$
The potential is minimised for $|\langle \vec \phi\rangle|^2 = v^2 = 6m^2/\lambda$. By an $SO(3)$ transformation, we can pick all the components of $\vec \phi$ to vanish except, $\langle \phi_3 \rangle = v$. You can check that $\langle \vec \phi \rangle$ is invariant under $$H = SO(2) \subset G = SO(3).$$
$SO(2)$ and $SO(3)$ differ in the number of generators by two, giving rise to two Goldstone bosons. We can expand the Lagrangian to see,
$$\mathcal L = -\frac14 (F^a_{\mu\nu})^2 + \frac{1}{4}g^2 A^a_\mu A^b_\mu \vec{v}^T \{\tau^a, \tau^b\}\vec{v}$$
and inserting the explicit generators $\tau^a$ one finds,
$$\mathcal L = -\frac14 (F^a_{\mu\nu})^2 + \frac12 m_A^2(A^1_\mu A^1_\mu + A^2_\mu A^2_\mu)$$
where we have identified a mass, $m_A^2 = g^2 v^2$. Thus, we find the theory possesses a massless gauge boson, $A^3$ as well as two massive gauge bosons, $A^{1,2}$. As you come from a mathematics background, remember to think of $A$ as a connection, taking values in the Lie algebra of the gauge group.
A final word: Since you state your own interests lie in Lie groups and smooth manifolds, if symmetry breaking is of interest to you, then you will probably find things like solitons, soliton moduli spaces and vacuum manifolds of interest mostly, and a good book is,
- J. Weinberg, Classical Solutions in Quantum Field Theory, Cambridge University Press.