Let's say you have a certain gauge theory, i.e., a QFT with gluons plus, perhaps, some other matter fields. Take for example one of such matter fields $\phi$. In general $\phi$ will couple to the gluons, i.e., it will interact with them. In fact, it is possible that $\phi$ only interacts with some of the gluons. Let's give these objects some names.
Take the algebra of the gauge theory to be $\mathfrak g$, i.e., the structure of $\mathfrak g$ determines the dynamics of the gluons. This algebra can be written as $\mathfrak g=\mathfrak g_\phi+\mathfrak h$, where $\mathfrak g_\phi$ is the subset of gluons that couple to $\phi$, and $\mathfrak h$ is the rest of gluons, i.e., those that do not couple to $\phi$.
If $\phi$ condenses, then the gluons in $\mathfrak g_\phi$ will feel a "drag", i.e., they will become massive. The rest of gluons, those in $\mathfrak h$, do not interact with $\phi$, and so they do not feel any drag: they stay massless. In this situation, one says that the algebra $\mathfrak g$ is broken down to $\mathfrak h$. The reason for this name is that the gluons of $\mathfrak h$ are massless, i.e., they can propagate through macroscopic distances, and they behave as regular gluons, like those of $\mathfrak g$ before condensation. The gapped gluons, those in $\mathfrak g_\phi$, are massive, and so their range is exponentially small: they are not seen at macroscopic distances.
Of course, the gapped gluons are still there: the symmetry is still $\mathfrak g$. But macroscopically we only see $\mathfrak h$, so the theory mostly looks like a gauge theory with algebra $\mathfrak h$. This is the reason we say $\phi$ has broken $\mathfrak g\to\mathfrak h$.
In the Standard Model, $\phi$ is the Higgs field. Roughly speaking, $\phi$ is charged under $\mathfrak{su}(2)$, but not under the other groups, so $\mathfrak g=\mathfrak{su}(3)+\mathfrak{su}(2)+\mathfrak u(1)$ is broken down to $\mathfrak h=\mathfrak{su}(3)+\mathfrak u(1)$, while the charged subgroup $\mathfrak g_h=\mathfrak{su}(2)$ becomes massive. The spectrum is not changed by the condensation: we still have $\mathfrak{su}(3)$ gluons, plus $\mathfrak{su}(2)$ gluons (also known as $W^\pm,Z$), and $\mathfrak u(1)$ gluons (also known as the photon). But those in the broken subgroup $\mathfrak{su}(2)$ are massive, and so they are only observable if you go to distances shorter than their wavelength, $1/m$. At macroscopic distances, the massive gluons are mostly invisible.
Of course, this is not the end of the story. For example, it is believed that mesons (quark bilinears) condense too, and those are charged under $\mathfrak{su}(3)$, so these become "massive" as well. Color gluons are not seen at macroscopic distances, by a very similar mechanism the $W^\pm,Z$ are not seen, the main difference is that the former corresponds to some dynamical (strong-coupling) condensation, while the latter to a kinematical (tree-level) condensation. Of course, the Higgs case is much better understood, as it can be seen in perturbation theory, while mesons and baryons are still poorly understood.
This leaves us only with $\mathfrak u(1)$, and indeed in real life we only see electromagnetism. We do not "feel" the weak and strong forces at macroscopic distances. So, to summarize: the full group of the Standard Model is $\mathfrak{su}(3)+\mathfrak{su}(2)+\mathfrak u(1)$, but due to the Higgs mechanism, the $\mathfrak{su}(2)$ part becomes massive, and disappears at long distances. By a more subtle mechanism, the $\mathfrak{su}(3)$ part also disappears, and we are left with the $\mathfrak u(1)$ part only.
