What exactly does it imply about a condensed matter system to have particle number conserved or not conserved?
For example, why does the superconducting phase break particle number conservation while the insulating phase does not? What exactly brings about the breaking of this conservation?
Conservation of the number of particle is a symmetry of the system. As Akshay Kumar said in his response, when the number of particles operator commutes with the Hamiltonian, it is conserved. It simply means, well, that's the number of particle is conserved. Particles are all what is discussed in condensed matter (better to say quasi-particles actually), like electrons and holes (certainly the most famous ones, but we should say quasi-particle of positive and negative excitation energy relative to the Fermi energy if we were not lazy: I think the length of their exact names is sufficient to keep electron and hole in the following :-). So it should be fine to know if some (quasi-)particles can or not pop-out from nowhere. Fortunately enough, when the particle number is conserved, they do not pop-out from nowhere, they can only be transmuted from an other (quasi-)particle. That's what happens with superconductivity: two electrons disappear and one Cooper pair emerges (in a really pictorial way of speaking).
Now for superconductivity, it is easier to say that you will conserve the number of particles if your Hamiltonian is invariant with respect to the transformation
$$c\rightarrow e^{\mathbf{i}\theta}c$$ and $$c^{\dagger}\rightarrow e^{-\mathbf{i}\theta}c^{\dagger}$$
where the $c$'s are the fermionic operators, and $\theta$ an angle. Actually, $\theta$ is better defined as the generator of the U(1) rotation. In particular, if your Hamiltonian (better to say a Lagrangian) is invariant with the phase shift operation defined above, you can associated a Noether current to it. For the U(1) rotation symmetry, the conserved current will be the current of particles. In particular for time independent problems (to simplify say), the number of particles will be conserved if your Hamiltonian is invariant under the above defined transformation.
The BCS Hamiltonian describing the conventional superconductivity reads (I discard the one body term and the spin for simplicity: they change nothing to the conclusions we want to arrive at)
$$H_{\text{BCS}}\propto c^{\dagger}c^{\dagger}cc$$
such that making the U(1) rotation does not change it, since there is the same number of $c$ than the number of $c^{\dagger}$ operators.
Below the critical temperature, the new superconducting phase appears, characterised by a non vanishing order parameter (i.e. the number of Cooper pair, still in a pictorial way of speaking-- better to say the superconducting gap parameter)
$$\Delta\propto cc$$
which transforms under a U(1) phase shift like
$$\Delta\rightarrow e^{2\mathbf{i}\theta}\Delta$$
since there are now two $c$ operators not compensated by some $c^{\dagger}$. So, the order parameter $\Delta$ is not invariant under the U(1) phase transformation symmetry. One says that the ground state of superconductivity does not conserve the number of particles.
Note that:
Saying that the number of particles is not conserved is an abuse of language, since the total number of electrons is the same in both the normal and superconducting phases. The condensed (superconducting) phase simply does not verify the invariance under the U(1) rotation. But that's true that some electrons are disappearing in a sense. As I said in the introduction: they are transmuted in Cooper pairs (once again, that's a pictorial way of speaking).
Such mechanism when the Hamiltonian verifies a symmetry that its ground state does not is called a spontaneous symmetry breaking. Superconductivity is just one example of such mechanism.
$\Delta$ remains invariant under the restricted rotation $c\rightarrow e^{\mathbf{i}n\pi}c$ with $n\in\mathbb{Z}$. Since there are only two such rotation elements $e^{\mathbf{i}n\pi}=\pm 1$, one says that U(1) has been broken to $\mathbb{Z}_{2}$ (a fancy notation for the group of only two elements).
Post-Scriptum: Please tell me if you need more explanations about some terminology. I don't know where you're starting from and my answer is a little bit abrupt for young students I believe.
Think of writing down the equation for the number operator N in the Heisenberg picture. Now if N commutes with the Hamiltonian H then the time derivative of N is 0, i.e. N is conserved.
