Classical electrodynamics as an $\mathrm{U}(1)$ gauge theory Preface: I haven't studied QED or any other QFT formally, only by occasionally flipping through books, and having a working knowledge of the mathematics of gauge theories (principal bundles, etc.).
As far as I am aware, the status of electrodynamics as an $\mathrm{U}(1)$ gauge theory comes from quantum-mechanical considerations, namely the ability to rotate the complex phase of a wave function describing a charged particle, however the "classical" gauge freedom $A\mapsto A+\mathrm{d}\chi$ follows naturally from this, and so does the $F$ electromagnetic field strength tensor as the $\mathrm{U}(1)$ connection's curvature, and this $F$ is the same $F$ as it is in classical ED.
My question is regarding if it is possible to formulate purely classical electrodynamics as an $\mathrm{U}(1)$ gauge theory, including motivation to do so (eg. not just postulating out of thin air that $A$ should be a $\mathrm{U}(1)$ connection's connection form, but giving a reason for it too)?
 A: Yes, you can formulate Maxwell's equations as a classical $U(1)$ gauge theory without using the word "quantum" at all. In non-relativistic notation, we introduce a scalar potential $\phi$ and a vector potential $\vec{A}$ which are related to the electric $\vec{E}$ and magnetic $\vec{B}$ fields via
\begin{eqnarray}
-\nabla \phi - \frac{\partial \vec{A}}{\partial t} &=& \vec{E} \\
\nabla \times \vec{A} &=& \vec{B}
\end{eqnarray}
From these definitions, it is easy to check that $\vec{E}$ and $\vec{B}$ are unchanged if we transform $\phi$ and $\vec{A}$ by
\begin{eqnarray}
\phi &\rightarrow& \phi - \frac{\partial \lambda}{\partial t}\\
\vec{A} &\rightarrow& \vec{A} + \nabla \lambda
\end{eqnarray}
for an arbitrary (smooth) function $\lambda$. This is gauge invariance. The above set of statements can be reformulated in much more sophisticated ways, including saying that $(\phi,\vec{A})$ is a connection on a $U(1)$ fiber bundle.
The potential formalism was known and used well before quantum electrodynamics was discovered. The main reason for introducing the potentials classically, is calculational convenience. Instead of needing to solve for the 6 components of $\vec{E}$ and $\vec{B}$, we only need to solve for 4 components (1 in $\phi$ and 3 in $\vec{A}$).
In fact it is even better than this because we can use gauge invariance to massage the form of the equations into standard forms. For instance, we can always choose the function $\lambda$ such that the sourced Maxwell's equations take the form of four decoupled standard wave equations (the Lorenz gauge)
\begin{eqnarray}
-\frac{1}{c^2} \frac{\partial^2 \phi}{\partial t^2} + \nabla^2 \phi &=& 4\pi \rho \\
-\frac{1}{c^2} \frac{\partial^2 \vec{A}}{\partial t^2} + \nabla^2 \vec{A} &=& 4\pi \vec{J}
\end{eqnarray}
where $\rho$ is the charge density and $\vec{J}$ is the current density.
These can be solved formally using Green's function methods, and the expressions for the electric and magnetic fields can be obtained by differentiating the potentials. This method of solution is easier than trying to directly solve for $\vec{E}$ and $\vec{B}$, because the sourced Maxwell's equations do not take the form of decoupled copies of the wave equation in standard form in those variables (the components are mixed together and there are a mix of different kinds of second spatial derivatives).
Also note that we need to solve fewer independent equations using the potential formalism. Since $\nabla \cdot \nabla \times \vec{A}=0$ identically, Maxwell's equation $\nabla \cdot B=0$ is automatically satisfied if we solve for the vector potential. Similarly, the equation $\nabla \times \vec{E} = - \frac{\partial \vec{B}}{\partial t}$ is automatically satisfied by the potential, which you can verify using the identity $\nabla \times \nabla \phi=0$ and the definition $\vec{B}=\nabla \times \vec{A}$.
Finally, as others have noted, using the potentials lets you express Maxwell's theory in terms of a local Lagrangian. This is useful at a formal level; for example it lets you derive a stress-energy tensor in a systematic way, and to explain conservation of charge as a consequence of Noether's theorem applied to the global $U(1)$ symmetry ($\lambda={\rm const}$, which leaves the potentials invariant but under which charged particles and fields will transform).

Incidentally, to make a comment that isn't directly in response to your question, but which I think is relevant: it's worth understanding why the formulation of a $U(1)$ gauge theory is useful in quantum theory. It actually is not necessary to formulate Maxwell's equations as a gauge theory quantum mechanically, but it is convenient, because the local Lagrangian formulation makes the locality of physical results manifest. However, this comes at the cost of needing to establish that the unphysical parts of the gauge field do not contribute to the final answer, established using an elaborate set of relations known as Ward-Takahashi identities, and makes unitarity difficult to establish. In other formulations (helicity-spinor variables), locality is not obvious but other properties of the theory such as unitarity are.
A: It is hard to imagine an argument that would convince OP, given the fact that classical $U(1)$ gauge theory need not be the way it is. There are perfectly good theories that are very similar but differ in some aspects that only matter quantum-mechanically.
For example, Maxwell theory can be formulated without a lagrangian: one can postulate the equations of motion directly. On top of that, it can be formulated directly in terms of the field strength. Also, compactness of the target space is necessary. Furthermore, you can formulate the theory in terms of gauge potentials but drop the requirement for gauge invariance entirely. Etc. None of these claims are true in the quantum theory:

*

*In QFT the lagrangian is very important, otherwise it is extremely hard to construct consistent theories (which satisfy covariance, locality, causality, etc.). The lagrangian is almost measurable (via the path integral), so it is not just a formal tool.


*In QFT, the interactions with matter can be written in a local way only through the gauge potential, $j_\mu A^\mu$. So you really need $A_\mu$. (Also, the holonomy is physical so again, $A_\mu$ is not just a formal tool).


*In QFT, if the target space is $\mathbb R$ instead of $U(1)$, things sometimes become nasty. (There are issues with the spectrum, also it cannot be completed to a gravitational theory, etc).


*In QFT, gauge invariance is essential, otherwise the theory makes no sense (you lose unitarity). A classical theory of a vector field $A_\mu$, with no condition on the longitudinal mode $\partial_\mu A^\mu$, is perfectly healthy, so we do not need to impose gauge invariance if we don't want to.
Also, I want to mention that the usual motivation for gauge invariance in QFT is just wrong. There is no reason a global symmetry should be local. Global symmetries are perfectly fine. Fields with an observable phase are perfectly fine. The common statement "relativistic invariance and locality imply that the phase cannot be measurable and therefore should be gauged" is just false.
Classical field theories with global $U(1)$ invariance are well-defined, so the usual argument could also be used in classical field theory. Therefore, there is nothing special about quantum mechanics here: if you find the quantum argument convincing (although it very much is not), then there is nothing stopping you from applying the same logic to classical field theory.
A: I'd like to add that electrodynamics was originally described classically via the Maxwell equations. These were all determined experimentally, and it was only found through careful study that these equations satisfy a gauge invariance (which we now know is associated with a $U(1)$ gauge group). Gauge invariance is accidental, and there's not much motivation for it classically other than the equations that were determined experimentally have it.
There is plenty of motivation for gauge invariance in the quantum theory, as described by @AccidentalFourierTransform .
Edit
As per @Filippo 's comment, I will elaborate. A gauge field with respect to some Lie group $G$ is a field $A_{\mu}$ valued in the algebra of $G$ which transforms under a local gauge transformation $g(x):\mathbb{R}^4\to G$ like $A_{\mu}\to g^{-1}A_{\mu}g-ig^{-1}\partial_{\mu}g$. Taking $G=U(1)$, $g(x)$ can be parametrized as $g(x)=\exp(i\alpha(x))$ for some function $\alpha:\mathbb{R}^4\to U(1)$ (note that $\alpha$ is defined modulo $2\pi$). Hence a $U(1)$ gauge field transforms as
$$A_{\mu}\to e^{-i\alpha}A_{\mu}e^{i\alpha}-ie^{-i\alpha}\partial_{\mu}e^{i\alpha}=A_{\mu}+\partial_{\mu}\alpha$$
Which is precisely what the vector potential in electromagnetism is invariant under. Note though that just looking at this equation, there is nothing that would indicate that $\alpha(x)$ is valued in $U(1)$. It could very well be valued in $\mathbb{R}$. Indeed from the classical perspective there is nothing that prevents $\alpha$ to be valued in $\mathbb{R}$, and it is only when you consider it quantum mechanically that you see the consequences of which choice of $\alpha$ you take. Whether the $U(1)$ gauge group in the Standard Model is compact (isomorphic to $S^1$) or non-compact (isomorphic to $\mathbb{R}$) is technically still an open problem, although as @AccidentalFourierTransform pointed out there are gravitational considerations which suggest the gauge group needs to be compact.
A: A gauge theoretic treatment of classical electrodynamics and Yang-Mills theory is done with the aid of Principal Lie group bundle morphisms. The following definitions and notations are borrowed from Frederic Schuller's lecture notes on the Geometrical Anatomy of Theoretical Physics and "Mathematical Gauge theory with Applications to the Standard Model of Particle Physics" by Mark J.D. Hamilton
$\textbf{Definition}:$ A smooth bundle $(P,\pi,M)$ is called a principal G-bundle if total space P is equipped with a right free G action and bundle morphisms :$$(P\xrightarrow\pi{}M)\simeq_{bdl}(P\xrightarrow\rho{}(P/G))$$ exists, where the quotient map $\rho$ takes each element $p\in P$ to the equivalence class (orbit) $G_p\subset P/G$. The set $G_p :=\{q\in P |\exists g\in G : q=g\circ p\}$ is diffeomorphic to G if the action $\circ :G\times P\to P$ is free. $P/G=\{G_p|p\in P\}$ is the orbit space of P. Physically, we can regard M as our smooth space-time manifold. To construct gauge theory for classical electrodynamics, assume $(P,\pi, M)$ as a vector bundle. This forms the basic foundation and now we can equip additional structures namely - connection 1-form (electromagnetic potentials) and Curvature 2-form (Maxwell tensor) satisfying Bianchi identity.
$\textbf{Connection 1-form}:$ The tangent space at each point $p\in P$ can be decomposed into horizontal and vertical subspace: $T_pP=H_pP\oplus V_pP$ where $V_pP :=$ker($\pi_*$)$_p$. Also, a bijection exists between vertical subspace and tangent space of G $$\iota_p :T_eG\to V_pP$$ where $T_eG$ is the tangent vector space at identity element $e\in G$. A connection on $(P,\pi,M)$ is a choice of horizontal space at each $p\in P$ and is conveniently encoded in Lie-algebra valued 1-form:$$\textbf{A}\:\colon \Gamma(TM)\to T_eG$$ For each point $p\in P$ and $\forall X\in T_pP$, $A_p(X):=\iota_p^{-1}($ver$(X))$. We see that horizontal space is essentiall y the kernel of the map , $H_pP=$ker($A_p$). $\Gamma(TM)$ is the set of all vector fields on M.
Locally (for an open subspace $U\subset M$), we can define $$A^U\:\colon\Gamma(TU)\to T_eG$$ where $A^U :=\sigma^*A$ ,$\:\sigma\in\Gamma(TU)$. At each point $p\in U$, we have $(A^U)_p=A^U_{\mu}(dx^{\mu})_p$ in some local basis ($\{\partial_{\mu}\}$) in U. We can also choose additional "index" corresponding to basis $\{e_a\}$ of the Lie algebra $\mathcal{g}$: $A^U_{\mu}=\sum (A^U)^a_{\mu}e_a$. These components are called local gauge boson fields. The local coordinate transformation $(\sigma_1,U_1)$ to $(\sigma_2,U_2)$ is given by the Gauge map:$$\Omega\:\colon U_1\cap U_2\to G$$ $\sigma_2(m):=\sigma_1(m)\circ\Omega(m)\:\:\:\forall m\in U_1\cap U_2$. The transformation rule is given by $$(A^{U_2})_{\mu}=\Omega^{-1}(A^{U_1})_{\mu}\Omega+\Omega^{-1}\partial_{\mu}\Omega$$
$\textbf{Curvature 2-form}:$ The curvature 2-form $\textbf{F}$ is defined as the exterior covariant derivative of connection 1-form $\textbf{A}$ :$$\textbf{F}=d\textbf{A}(hor(\: -\:))=d\textbf{A}+\frac{1}{2}[\textbf{A},\textbf{A}] $$ Similar to connection 1- form, the local curvatures can be defined as $F^U :=\sigma^*\textbf{F}$ and the local coordinate transformation under Gauge map is $$(F^{U_2})_{\mu}=\Omega^{-1}(F^{U_1})_{\mu}\Omega$$ For classical electrodynamics, we take $G=U(1)$. So all elements of G are of the form $e^{i\theta}$ and it is abelian. We see that connection 1-form $\textbf{A}$ transforms under the Gauge map as $$A_{\mu}\:\to e^{-i\theta}A_{\mu}e^{i\theta}+e^{-i\theta}\partial_{\mu}e^{i\theta}=A_{\mu}+i\partial_{\mu}\theta$$ This is essentially the gauge transformation of 1st kind. We can also see that the curvature  2-form takes the form of Maxwell tensor :$$\textbf{F}(\partial_{\mu},\partial_{\nu})=d\textbf{A}(\partial_{\mu},\partial_{\nu})+\frac{1}{2}[A_{\mu},A_{\nu}]=2\partial_{[\mu}A_{\nu]}$$ and it is invariant under the Gauge map. The lagrangian density for EM field can be formulated as: $$\mathcal{L}=-\frac{1}{2}\langle F^A_M, F^A_M\rangle_{Ad(P)}$$ for connection A It is a smooth real valued function on $M$ which can be extremized to give free Maxwell's equation or Yang-Mills field equation for general $SU(N)$ Lie group. Here $\langle - , -\rangle$ is the Ad() invariant scalar product on the Lie algebra.
A: The pertinent thing about $U(1)$ is that it is abelian. Morally, it is the "only" compact Lie group. Now, if you're going to  create a gauge field, why use $U(1)$?
Well, the thing is, there is no paricular reason to use $U(1)$. The full gauge group of the standard model is $SU(3) \times SU(2) \times U(1)$. The thing is, the theories built out of non abelian gauge groups behave very differently than the ones built out of non abelian ones. The non-abelian nature of $SU(3)$, the strong force (QCD), makes its force carriers, gluons, interact with each other. This has many properties, including the famous "mass gap." A single gluon propagating through empty space is simply not a state which exists in QCD.
However, the classical Lagrangian of QCD (as an $SU(3)$ Yang Mills theory) can have massless 'gluon waves' travelling through space. So here we see that the non abelian nature of the group makes the quantum mechanical story qualitatively different from the classical story. In fact, the strong interacting nature of the quantum mechanics story prevents you from easily seeing QCD at a microscopic level. This is why it took people so much longer to discover QCD than E&M.
TLDR; because $U(1)$ is abelian, photons do not interact with each other, which means that there are "free" single photon states in the theory. Therefore the classical theory is qualitatively pretty similar to the quantum theory, which is what allowed classical E&M to be discovered and formulated, and for you to ask this question in the first place.
(Another way to say this it that E&M is a "long range" force, and easily discoverable, because the photon is massless.)

Another thing to address: why $U(1)$ and not $\mathbb{R}$? Within the context of the standard model, the $U(1)$ arises because it's a subgroup of $SU(2) \times U(1)$ left unbroken by the Higgs mechanism. Because $SU(2) \times U(1)$ is compact, it has only compact subgroups. So there are no $\mathbb{R}$'s around. Now you might ask, why couldn't we use a subgroup of a non compact non abelian group like $SO(2,1)$? The problem with non compact non abelian groups is that their Killing form isn't positive definite, which ruins the quantum theory.
A: Classical electrodynamics can be formulated in a Lagrangian formalism without gauge invariance. Classical electromagnetic gauge invariance is not a principle of nature. I have show this in this paper: https://arxiv.org/abs/physics/0106078, published as Eur. Phys. J. D, vol. 8, p 9-12 (2000).
Once you have the decoupled standard wave equations you realize that these constitute a one-to-one relation between source and potential. Then you realize that the Lorenz condition is just the image of current conservation. Add to this that these equations are the euler-lagrange equation of the Fermi lagrangian and the non gauge invariant formation of electromagnetism emerges.
A: CED can be formulated exclusively in terms of electric and magnetic fields, so no motivation for $U(1)$ exists. Any varyable change then is a matter of convenience rather than some necessity.
