1) Postponing for a moment the issue of magnetic monopoles, one conventional answer is, that the gauge potential $A_{\mu}$ (as opposed to, e.g., the electric and magnetic $\vec{E}$ and $\vec{B}$ fields) constitute the true fundamental variables and (the photon field) of QED.
At the classically level, by saying that $A_{\mu}$ are fundamental variables, we mean that the Maxwell action $S[A]$ depends on $A_{\mu}$, and that the corresponding Euler-Lagrange equations are the remaining Maxwell equations, namely Gauss' and (modified) Ampere's laws. (The obsolete Maxwell equations are Bianchi identities, which are rendered vacuous by the existence of the gauge potential $A_{\mu}$.)
Quantum mechanically, it seems appropriate to mention the Aharonov Bohm effect, which seem to indicate that the $A_{\mu}$ gauge field has physical consequences not already encoded in the gauge-invariant $\vec{E}$ and $\vec{B}$ fields. (However, check out this and this Phys.SE posts.)
2) Up until now we haven't discussed magnetic monopoles and Dirac strings.
At a Dirac magnetic monopole, the $A_{\mu}$ gauge potential is not well-defined. This is the main topic of, e.g., this, and this questions.
Dirac magnetic monopoles are usually dismissed, but there are other types of magnetic monopoles, namely the (generalized) 't Hooft-Polyakov magnetic monopole. See also this Phys.SE post.
Although (generalized) 't Hooft-Polyakov magnetic monopoles have so far not been experimentally detected, there are various theoretical reasons to believe they do exist, see e.g. this, this, and this questions.