If gauge symmetries are fake, then why do we care if they are anomalous? My understanding is that gauge symmetries are fake in that they are only redundancies of our description of the system that we put in (either knowingly or unknowingly) see Gauge symmetry is not a symmetry?. Moreover it seems like we could in principal work without the gauge redundancy, and things would just be a bit nastier at intermediate stages of a calculation. At the same time, people put a great deal of emphasis that gauge symmetries cannot be anomalous. I understand that if we work with the gauge redundancy then we do indeed need it to be anomaly free so that the unphysical polarizations will cancel. However, what if we work without the gauge redundancy, with just the physical polarizations, what goes wrong then if we don't make sure the would-be gauge symmetry is not anomalous?
 A: If gauge symmetries are not fake, but are real symmetries,
then， when they are anomalous, it simply means that the
theory just does not have the symmetries.
The theory is still well defined, at least.
If gauge symmetries are fake and representing  redundancy in our description,
then， when they are anomalous, it means that the
theory is inconstant.
So because gauge symmetries are fake, that makes us do care if they are anomalous?
A: Related Why do some anomalies (only) lead to inconsistent quantum field theories

What if we work without the gauge redundancy, with just the physical
  polarizations, what goes wrong then if we don't make sure the would-be
  gauge symmetry is not anomalous?

This is a very natural question I wondered some time ago. 
In electrodynamics, if you want to work just with two physical polarizations, you have two possibilities:


*

*Construct the theory only with gauge invariant fields such us $F_{\mu\nu}$. However, you cannot recover the classical Maxwell equation from a variational principle and moreover the S-matrix does not reflect the $1/r$ electrostatic potential. (Furthermore, you would need more general gauge invariant objects to reproduce the Aharanov-Bohm experiment.)

*Construct the theory with a field $A_\mu$ made up creation/annihilation operators of the two physical transverse polarizations. In this case the $A_\mu$ does not transform as a Lorentz four vector and you need gauge invariance to kill the non-covariant terms (if you want to preserve Poincare invariance).


Is it then impossible to work without gauge invariance? I would just say that we do not know how to do it, and it seems difficult. 
In short, to answer the OP quoted question, what goes wrong is either the loss of Poincare invariance or the $1/r$ electrostatic potential.  
