That's a nice question. The cleanest answer (that I know of) relies on a result of Milnor [1].
But first, let's rephrase things a bit. For a group $G$, let's define a flat $G$-bundle to be a principal $G$-bundle with a flat connection.
Your physics question translates to the following question in mathematics
If $G$ is a discrete group, why is every principal $G$-bundle flat?
Milnor's characterisation of flat bundles [1] states that
[Milnor's criterion] A principal $G$-bundle on $X$ is flat if and only if it is induced from
the universal covering bundle of $X$ by a homomorphism $\pi_1(X)\to G$.
So now, let's show that $G$-bundles with discrete structure group satisfy Milnor's criterion.
Suppose you want to build a $G$-gauge theory over $X$, i.e. you want to construct a principal $G$-bundle over $X$. Such a bundle is classified by a map $\newcommand{\B}{\mathrm{B}}$
$$ X\longrightarrow\B G,\tag{1}$$
where $\B G$ is the classifying space of $G$. In other words, there is a principal $G$-bundle $\xi:\mathrm{E}G\to \B G$$(*)$, such that all principal $G$-bundles on $X$ are pullbacks of $\xi$. A nice explanation of this is given in this math.SE answer.
Oh, but now we're in business! For a discrete group, $\B G \cong K(G,1)$ is an Eilenberg-MacLane space, so it holds that $\pi_1(\B G)\cong G$, where $\pi_1$ is the first homotopy group. Cool! So I can take $\pi_1$ of (1) and say that for a discrete group all principal $G$-bundles are classified by a map
$$ \pi_1(X)\longrightarrow G.\tag{2}$$
Therefore all principal $G$-bundles for a discrete group $G$ satisfy Milnor's criterion and are, therefore, flat.
In contrast, this wouldn't work for a continuous, say, group, because $\B G$ would not be an Eilenberg-MacLane space and it would not, generally, hold that $\pi_1(G)\cong G$.
To make some connection (pun not really intended) with physics, note that, for a discrete group the set of maps (2): $\operatorname{Hom}(\pi_1(X),G)$ is isomorphic to the first cohomology group $\mathrm{H}^1(X;G)$ (see e.g. this phys.SE answer of mine), therefore elements of $\operatorname{Hom}(\pi_1(X),G)$ define connection one-forms with zero curvature, i.e. flat connection one-forms, $A\in\mathrm{H}^1(X;G)$, which is how we usually think of gauge fields of discrete groups in physics.
Finally, there is another way to show the same result, by using the constancy of transition functions and tangency of the connection to an integrable manifold as explained in this MathOverflow answer.
References: [1] John Milnor, On the existence of a connection with curvature zero , dx.doi.org/10.5169/seals-25344
$(*)$$\mathrm{E} G$ is a weakly contractible space, defined in the Wikipedia article linked above
