General question about gauge invariance of Lagrangian. Illustration with $U(1)$ I am learning gauge theories and I am a little confused with things.
I would like to take the simplest example : the $U(1)$ invariance to check if I understood the basics.
When we work with electromagnetism, we know that we have a gauge invariance :
$$ A_\mu \rightarrow A_\mu + \partial_\mu \alpha $$
The E.O.M are unchanged if we change like this the potential vector.
So, what we want is a Lagrangian invariant under such transformation. In practice, the E.M lagrangian :
$$ \mathcal{L}_{EM}=F_{\mu \nu}F^{\mu \nu} $$
is invariant under thoose gauge transformations.
Now, we want to couple this field to the matter, so the most general Lagrangian we write is :
$$ \mathcal{L}=F_{\mu \nu}F^{\mu \nu}+\partial_\mu \phi^* \partial^\mu \phi + \mathcal{L}_{int}(\phi, A)$$
As the electromagnetism is gauge invariant, the total Lagrangian must also follow this symmetry. Thus $\mathcal{L}_{int}(\phi, A)$ must be gauge invariant.
Finally, the simplest Lagrangian that follows the gauge invariance is :
$$ \mathcal{L}=F_{\mu \nu}F^{\mu \nu}+D_\mu \phi^* D^\mu \phi$$
Where $D^{(A)}_{\mu}=\partial_\mu + ig A_\mu $.
And we know that under a group transformation $g \in U(1)$, we have by construction :
$$D^{(A')}_{\mu} (g \phi) = g D^{(A)}_{\mu}(\phi) $$.

First question : Link between $U(1)$ symmetry and vector potential.
If we say that we have $U(1)$ symmetry here, it is because to change the vector potential $ A_\mu \rightarrow A_\mu + \partial_\mu \alpha $ is equivalent to act on $\phi$ with an $U(1)$ element : $e^{-ig \alpha(x)}$.
So finally, to talk about $U(1)$ invariance of this Lagrangian, or gauge invariance of the potential vector $A_\mu$ is just the same thing said in two different ways.
Also, it is by construction a local gauge symmetry because $\alpha$ can of course depend on space.
Am I right ?
Second question : 
If I am right with what is written above, to try to write an $U(1)$ gauge invariant Lagrangian makes sense because it is linked to the gauge symmetry of electromagnetism (that we know well with maxwell equations).
But in gauge theories we also look for $SU(N)$ symmetries. How do we know from a physical point of view that it is the "good" group of symmetries ? For $U(1)$ it can be seen (if I am right with what I wrote) because of the gauge invariance of electromagnetism that we know from maxwell equation. But for QCD for example what is the intuition behind knowing it is the $SU(3)$ group of symmetry ? 
Did theoretician just got their inspiration for electromagnetism and wondered what would happen if we write lagrangian following more general symmetries. They tried with $SU(2)$ and $SU(3)$ and saw that it indeed fitted to some experiments ? (I'm almost sure it wasn't like that but this paragraph is to try to explain better my second question).
 A: I think there may be some confusion, or at least things may be made more precise. You say that,

So finally, to talk about $U(1)$ invariance of this Lagrangian, or
  gauge invariance of the potential vector $A_\mu$ is just the same
  thing said in two different ways.

The quantity of interest which we are checking for a symmetry (up to a total derivative) is the Lagrangian, such that the action is truly invariant. The Lagrangian itself depends on the potential $A_\mu$, so naturally if we say the Lagrangian has a $U(1)$ invariance, if we are to transform $\mathcal L$ then we have to define some way the group acts, which in this instance is on $A_\mu$.
It is totally analogous to for example, the $SU(2)$ symmetry of a function, $f(\mathbf x)$. The natural definition of symmetry would be to say that, $f(g^{-1}\mathbf x) = f(\mathbf x)$ for some $g \in SU(2)$, for example.


Also, it is by construction a local gauge symmetry because $\alpha$ can of
  course depend on space.

It is redundant to say, local gauge symmetry. By definition, if a transformation is local, i.e. depends on space-time coordinates, then it is a gauge transformation.
To say local gauge symmetry is like saying gauge gauge symmmetry. So, if a symmetry is global, it is simply a global symmetry and if it is local, it is a local symmetry or equivalently gauge symmetry.
Remember also that despite the name, a gauge symmetry should be thought of as a redundancy in our formulation of a theory, contrary to a global symmetry which leads to conservation laws.
Your second question should be posted separately, thought off the top of my head I believe there are related or duplicate questions to it.
