Gauge symmetry is not a symmetry? I have read before in one of Seiberg's articles something like, that gauge symmetry is not a symmetry but a redundancy in our description, by introducing fake degrees of freedom to facilitate calculations.
Regarding this I have a few questions:


*

*Why is it called a symmetry if it is not a symmetry? what about Noether theorem in this case? and the gauge groups U(1)...etc?

*Does that mean, in principle, that one can gauge any theory (just by introducing the proper fake degrees of freedom)?

*Are there analogs or other examples to this idea, of introducing fake degrees of freedom to facilitate the calculations or to build interactions, in classical physics? Is it like introducing the fictitious force if one insists on using Newton's 2nd law in a noninertial frame of reference?   

 A: In order:


*

*Because the term "gauge symmetry" pre-dates QFT. It was coined by Weyl, in an attempt to extend general relativity. In setting up GR, one could start with the idea that one cannot compare tangent vectors at different spacetime points without specifying a parallel transport/connection; Weyl tried to extend this to include size, thus the name "gauge". In modern parlance, he created a classical field theory of a $\mathbb{R}$-gauge theory. Because $\mathbb{R}$ is locally the same as $U(1)$ this gave the correct classical equations of motion for electrodynamics (i.e. Maxwell's equations). As we will go into below, at the classical level, there is no difference between gauge symmetry and "real" symmetries.

*Yes. In fact, a frequently used trick is to introduce such a symmetry to deal with constraints. Especially in subjects like condensed matter theory, where nothing is so special as to be believed to be fundamental, one often introduces more degrees of freedom and then "glue" them together with gauge fields. In particular, in the strong-coupling/Hubbard model theory of high-$T_c$ superconductors, one way to deal with the constraint that there be no more than one electron per site (no matter the spin) is to introduce spinons (fermions) and holons (bosons) and a non-Abelian gauge field, such that really the low energy dynamics is confined --- thus reproducing the physical electron; but one can then go and look for deconfined phases and ask whether those are helpful. This is a whole other review paper in and of itself. (Google terms: "patrick lee gauge theory high tc".)

*You need to distinguish between forces and fields/degrees of freedom. Forces are, at best, an illusion anyway. Degrees of freedom really matter however. In quantum mechanics, one can be very precise about the difference. Two states $\left|a\right\rangle$ and $\left|b\right\rangle$ are "symmetric" if there is a unitary operator $U$ s.t. $$U\left|a\right\rangle = \left|b\right\rangle$$ and $$\left\langle a|A|a\right\rangle =\left\langle b|A|b\right\rangle $$ where $A$ is any physical observable. "Gauge" symmetries are those where we decide to label the same state $\left|\psi\right\rangle$ as both $a$ and $b$. In classical mechanics, both are represented the same way as symmetries (discrete or otherwise) of a symplectic manifold. Thus in classical mechanics these are not separate, because both real and gauge symmetries lead to the same equations of motion; put another way, in a path-integral formalism you only notice the difference with "large" transformations, and locally the action is the same. A good example of this is the Gibbs paradox of working out the entropy of mixing identical particles -- one has to introduce by hand a factor of $N!$ to avoid overcounting --- this is because at the quantum level, swapping two particles is a gauge symmetry. This symmetry makes no difference to the local structure (in differential geometry speak) so one cannot observe it classically.
A general thing -- when people say "gauge theory" they often mean a much more restricted version of what this whole discussion has been about. For the most part, they mean a theory where the configuration variable includes a connection on some manifold. These are a vastly restricted version, but covers the kind that people tend to work with, and that's where terms like "local symmetry" tend to come from. Speaking as a condensed matter physicist, I tend to think of those as theories of closed loops (because the holonomy around a loop is "gauge invariant") or if fermions are involved, open loops. Various phases are then condensations of these loops, etc. (For references, look at "string-net condensation" on Google.)
Finally, the discussion would be amiss without some words about "breaking" gauge symmetry. As with real symmetry breaking, this is a polite but useful fiction, and really refers to the fact that the ground state is not the naive vacuum. The key is commuting of limits --- if (correctly) takes the large system limit last (both IR and UV) then no breaking of any symmetry can occur. However, it is useful to put in by hand the fact that different real symmetric ground states are separately into different superselection sectors and so work with a reduced Hilbert space of only one of them; for gauge symmetries one can again do the same (carefully) commuting superselection with gauge fixing.
A: The (big) difference between a gauge theory and a theory with only rigid symmetry is precisely expressed by the Noether first and second theorems:
While in the case of a rigid symmetry, the currents corresponding to the group generators are conserved only as a consequence of the equations of motion, this is called that they are conserved "on-shell". In the case of a continuous gauge symmetry, the conservation laws 
become valid "off-shell", that is independently of the equations of motion. 
This implies for example the conservation of the electric charge irrespective of the equation of motion.
Now, the conservation law equations can be used in principle to reduce the number of fields.
The procedure is as follows: 


*

*Work on the subspace of the field configurations satisfying the conservation laws. However, there will still be residual gauge symmetries on this subspace. In order to get rid of those:

*Select a gauge fixing condition for each conservation law.
This will reduce the "number of field components" by two for every gauge symmetry.
The implementation of this procedure however is very difficult, because it actually requires to solve the conservation laws, and moreover, the reduced space of field configurations is very complicated.
This is the reason why this procedure is rarely implemented and other techniques like BRST are used.
A: When talking about symmetry, one should always indicate: symmetry of what?
If I measure the length of a stick in inches and then in centimeters, i.e. in different gauges, then I get two different answers, although the stick is the same in both cases. Similarly, when I measure the phase of a sine wave with two clocks that have different phases, then I get two different phases, and phase shifts form the group U(1). In the first example the stick is invariant under the change of gauge from centimeters to inches, but this has nothing to do with any physical symmetry of the stick. Noether's theorem has to do with symmetries of the Lagrangian. E.g. if the Lagrangian has spherical symmetry, then total angular momentum is conserved. The Noether theorem obviously also applies to quantum systems. A change of gauge is not a physical transformation, that is all. In quantum field theory one starts with a simple Lagrangian (e.g. Dirac Lagrangian), and then changes it so that it becomes invariant under local gauge changes, i.e. one then changes the derivative in the Dirac equation into a D which has a "gauge field" in it: to make this sound cryptic, one then says that "local gauge invariance has generated a gauge field", although this is not true. Imposing local gauge invariance simply puts a constraint on what sort of Lagrangians can be written. It is similar to demanding that a function F(z) be analytic in the complex plane, this also has serious consequences.
A: Gauge symmetry imposes local conservation laws, which are called Ward Identities in QED and Slavnov-Taylor identities for non-Abelian gauge theories. Those identities relate amplitudes or limit them. 
An example of those constraints imposed by gauge symmetry is the transversality of the vacuum polarization. To be more precise, gauge symmetry does not allow for a mass term for a photon on the Lagrangian. Yet, this could develop through quantum fluctuations. This is not happening due to the Ward identity that imposes transversality of the photon vacuum polarization. Another example is the relation between fermion propagator and the basic vertex in QED. It guarantees the absence of longitudinal photons. 
The idea is thus that gauge symmetry does impose a sort of Noether theorem, but in much more refined way. It shows up at the level of quantum corrections and limits them. These relations are, furthermore, local. They become a sort of local version of Noether theorem.
A: 1) Why is it called a symmetry if it is not a symmetry? what about Noether theorem in this case? and the gauge groups U(1)...etc?
Gauge symmetry is a local symmetry in CLASSICAL field theory. This may be why
people call gauge symmetry a local symmetry. But we know that our world is quantum.
In quantum systems, gauge symmetry is not a symmetry, in the sense that the gauge transformation does not change any quantum state and is a do-nothing transformation.
 Noether's theorem is a notion of classical theory. Quantum gauge theory (when described by the physical Hilbert space and Hamiltonian) has no 
Noether's theorem. 
Since the gauge symmetry is not a symmetry, the gauge group
does not mean too much, in the sense that two different gauge groups can sometimes
describe the same physical theory. For example, the $Z_2$ gauge theory
is equivalent to the following $U(1)\times U(1)$ Chern-Simons gauge theory:
$$\frac{K_{IJ}}{4\pi}a_{I,\mu} \partial_\nu a_{J,\lambda} \epsilon^{\mu\nu\lambda}$$ with $$K= \left(\begin{array}[cc]\\ 0& 2\\ 2& 0\\ \end{array}\right)$$ in (2+1)D. 
Since the gauge transformation is a do-nothing transformation and the gauge group is unphysical, it is better to describe gauge theory without using
gauge group and the related gauge transformation. This has been achieved by string-net theory. Although the string-net theory is developed to describe topological order, it can also be viewed as a description of gauge theory without using gauge group.
The study of topological order (or long-range entanglements) shows that
if a bosonic model has a long-range entangled ground state, then the low energy effective theory must be some kind of gauge theory. So the low energy effective
gauge theory is actually a reflection of the long-range entanglements in the ground state. 
So in condensed matter physics, gauge theory is not related to geometry or curvature. The gauge theory is directly related to and is a consequence of the long-range entanglements in the ground state. So maybe the gauge theory in our vacuum is also a direct reflection of the  long-range entanglements in the vacuum.
2) Does that mean, in principle, that one can gauge any theory (just by introducing the proper fake degrees of freedom)? 
Yes, one can rewrite any theory as a gauge theory of any gauge group.
However, such a gauge theory is usually in the confined phase and the effective theory at low energy is not a gauge theory.
Also see a related discussion:
Understanding Elitzur's theorem from Polyakov's simple argument?
