What is the more fundamental quantity? The electromagnetic field $F$ or the potential $A$? Read somewhere that potential is a more fundamental quantity than EM field because if the latter is more fundamental then gauge transformation will reduce to nothing more than a mathematical trick. I am thinking it another way. Please point out whether my assumption is right or wrong. Since the action principle is the most fundamental of all, anything that is involved in it has to be more fundamental. 4-potential is involved in the action integral in the term describing the interaction of the particle and the field. So doesn't that make it in that sense more fundamental? After all EM field tensor itself is constructed from the potential term in the EOM. 
 A: Independently of Quantum Mechanics, if you believe in the variational method, then the $A$ field is more fundamental than the $\vec{E}, \vec{B}$ fields since the Lagrangian cannot give the Maxwell equations without the 4-potential. You need the potentials to make a variation of the fields that will give you their equations of motion (i.e. Maxwell's equations).
Also, even at the classical level, Statistical Mechanics needs a Hamiltonian, which asks for the potentials. How would we do Classical Statistical Mechanics without the electromagnetic potentials?
A: There is no definition of "more fundamental" so no answer that is unequivocally correct here. 
I note that most answers have nevertheless opted for the potentials over the fields. But one should notice that this is in part owing to a feeling that Lagrangians offer the royal route to the physics. So it depends a bit on how one likes to construct ones field theory in the first place. You don't have to start from a Lagrangian.
It was rightly commented that the Aharonov-Bohm effect is itself a gauge-invariant effect so this effect does not on its own make a self-evident case that the potential is 'there' equally as directly as the fields are 'there' (i.e. manifesting themselves by acting as the cause of physical effects). The non-zero line integral of $\bf A$ requires that there be a $\bf B$ field somewhere. (The situation is comparable to looking at parallel transport on the surface on a cone: everywhere you go (avoiding the apex) is flat, yet there is a net rotation if the path encircles the apex---and this only happens if you do indeed have an apex to encircle, i.e. a cone not a cylinder. Similarly the $\bf B$ field has to be there for the AB effect.)
So I have no answer to 'which is more fundamental' but one can say that the potentials are making themselves 'felt' via their gradients (div and curl) and this makes them elusive. 
A: That the vector potential A of the electromagnetic field is not just a mathematical trick and has some physical significance has been demonstrated experimentally by the Aharonov-Bohm effect. Formerly the opinion prevailed that the the vector potential was a mathematical convenience to derive the electric amd magnetic fields.
A: I think it becomes clear that the $4$-potential is the more fundamental quantity when one views the issue from a geometric perspective. It can be said that $A$ is a $\mathfrak{u}(1)$-valued connection one-form,$^\dagger$ and in this case, $F=dA$ has the interpretation of the curvature; it is after all a commutator of derivatives.
This view of $F$ as curvature is even more apparent when examining gauge anomalies which turn out to be proportional to Chern classes involving $F$, and the Chern class of the tangent bundle of a manifold is indeed in terms of the curvature two-form. 
Analogously in GR, one thinks of the metric $g_{ab}$ as more fundamental than any of the curvature tensors one may derive from it; after all one would not say $R_{ab}$ is more fundamental.

$\dagger$ Being more precise than most physics textbooks, the $A$ you work with in physics is actually the pull-back of the connection by a section.
A: The vector potential is necessary for describing QED. You might wonder if you can get away with writing your theory only in terms of the trivially gauge invariant tensor $F_{\mu\nu}$. The problem with that is that photons are spin one particles, and I don't know of a way to quantize $F_{\mu\nu}$ that makes just spin 1 degrees of freedom propagate. On the other hand it is not (too) hard to make a theory of spin one particles from a vector field $A_\mu$.
As another complication, if $F_{\mu\nu}$ is to annihilate asymptotically free particles then it must have dimension 2, and so it has no renormalizeable couplings with fermions which would make writing down any sensible QFT problematic.
A: The following paper (I am one of the authors) explain the foundamental role of electromagnetic four potential: 
Maxwell's equations and Occam's razor
from the abstract:
"In this paper a straightforward application of Occam's razor principle to Maxwell's equation shows that only one entity, the electromagnetic four-potential, is at the origin of a plurality of concepts and entities in physics"
A: You cannot arrive at the Aharonov Bohm effect using the field tensor. It is zero in the domain of interaction. You need the potential. Hence the potential is the physical quantity. 
More generally, in Lagrangian mechanics  the potential plays the role of the coordinate. It is not possible to derive Maxwell's equations from an action principle using the force field as a coordinate.
I invite a anyone interested in this fundamental issue to read my peer reviewed paper at https://arxiv.org/abs/physics/0106078.
