I am reading material related to statistical physics, and I am having a problem understanding why we care about the notions of distinguishably and indistinguishability. I have found What are distinguishable and indistinguishable particles in statistical mechanics? on here, but it doesn't answer my question since it doesn't really discuss why we care. I have also seen Classical, identical particles which are distinguishable, which explains that we can't 'watch' quantum mechanical particles, but this doesn't really fix my confusion either since I don't really understand mathematically what 'watch' actually means. I will explain my confusion here.
My understanding is that we say particles are indistinguishable if there is no way, even in principle, to tell them apart, and distinguishable otherwise. I can understand how we might not be able to tell particles apart, but I can't see why this would ever be important. We can derive the partition function for (distinguishable, I think) particles basically by making a combinatoric argument about how many ways there are to partition $N$ particles into $k$ groups (states or energy levels), with these groups having sizes $n_1, \ldots, n_k$, finding that there are $\displaystyle\frac{N!}{\Pi_i n_i !}$ ways of doing this.
I don't see how not being able to tell the particles apart actually changes anything. As far as I can see, counting arguments like this don't rely on the ''indistinguishable'' and ''particle'' having any meaning whatsoever: $\displaystyle\frac{N!}{\Pi_i n_i !}$ is the number of ways of partitioning a set of size $N$ into $k$ sets with sizes $n_i$, regardless of what the elements of that set are, and whether we can ''tell them apart''.
I hope I've explained my confusion sufficiently: in a nutshell, it could be summarised as "given that counting arguments don't depend on distinguishability, why does having indistinguishable particles change anything at all?". My suspicion is that I am mistaken in believing this: certainly if we say a set has size $N$ we are assuming that it has $N$ 'distinct' elements, in some sense. I really can't see how this is the same notion of distinguishability as applies to particles, though, so I am quite stuck.
If anyone could help me out here, that would be great! Just to restate what I'm after: I feel that I understand what people mean by "indistinguishable", I just can't see why it would matter in terms of counting arguments.