Fields are sections of a bundle associated to a $\mathrm{SO}(1,3)$-bundle or to a gauge group bundle? In Quantum Field Theory particles are associated to unitary representations of the Poincare group and fields are classified according to the irreducible representations of the Lorentz group.
In the case of fields, every irreducible representation of the Lorentz group is characterized by $(j_1,j_2)\in \frac{1}{2}\mathbb{Z}^+$. This leads to the following situation:


*

*Since a field must take value on a representation space of some irreducible representation of the Lorentz group, it seems natural that fields must be sections of the associated bundle to the principal frame bundle $P_{SO_e^+(1,3)}M$ over spacetime characterized by the representation $(j_1,j_2)$ in question. This means that for each $(j_1,j_2)$ we would construct the associated bundle to the frame bundle related to this representation and the fields would be sections of said bundle.

*On the other hand, there is gauge invariance. When we have gauge invariance we have a covariant derivative to couple fields to the gauge field. Let's take QED as example for simplicity. The charged fields that interact with the EM field, must transform in a particular way under gauge transformations. In other words the gauge group $U(1)$ acts upon them. For a charged scalar field $\phi$ for instance, we must have that $$\phi(x)\mapsto e^{-i\alpha(x)}\phi(x).$$
In this case, it seems that the fields must be section of a bundle associated to a principal $U(1)$ bundle. It is not clear what this principal bundle is (the trivial bundle is always available, but it doesn't seem physically motivated).
Furthermore, a gauge covariant derivative acts upon such fields. Again, this can only be if these fields are sections of a bundle associated to a principal $U(1)$ bundle with one connection.

*There's furthermore the potential itself $A$. Although it starts as a field in the sense of (1) above, it ends up being a gauge-dependent representative of a connection 1-form on the principal bundle described in (2). If that is true, then it should be a section of the bundle $E\otimes  T^\ast M$ with $E$ the trivial bundle $M\times \mathfrak{u}(1)$.
Now my question is: how do we reconcille these three things? First the fields must be sections of some bundle associated to the principal bundle of orthonormal frames, but with gauge theory they end up needing to be sections of a bundle associated to a principal bundle, which is not even clear which principal bundle it is. Finally the gauge field ends up being a connection one-form, so it is a section of a totally different bundle.
Roughly speaking the issue is that (1) suggests fields are sections of one bundle, (2) suggest they are of another, and they can't be sections of both at the same time.
How all these things can be true at the same time? 
 A: The fields are sections of an associated bundle $E \xrightarrow{\pi_E} M$ over a manifold $M$ with a structure group $G$, which is for example $\operatorname{SU}(3)$ in the case of QCD.
The group of all gauge transformations $\operatorname{Gau}(E)$ is a group of bundle automorphisms, i.e., it does not change the structure of the bundle.  These transformations work fiberwise and project into the identity diffeomorphism of the base space. For example for a (non gauge) vector field this transformation takes the form:
$$V_{\mu}^{'A}(x) = \pi(g(x))^A_B V_{\mu}^B(x)$$
where $\pi$ is the structure group representation on $E$. This transformation is pointwise and does not act on the space time indices. It is called an internal automorphism.
However, these transformations do not exhaust the bundle automorphisms. For a class of bundles called natural bundles, more general transformations preserve the structure of the bundle:
$$V_{\mu}^{'A}(x') = \frac{\partial_{\mu}x'}{\partial_{\nu}x}\pi(g(x))^A_B V_{\nu}^B(x)$$
This transformation includes both a nontrivial diffeomorphism as well as a gauge transformation, and it projects to a nontrivial diffeomorphism on the base manifold: $x \rightarrow x'$.
The above shows that the group of all bundle automorphisms $Aut(E)$ decomposes according to the following exact sequence:
$$0 \rightarrow \operatorname{Gau}(E) \rightarrow \operatorname{Aut}(E) \rightarrow \operatorname{diff}(M)\rightarrow 0$$ 
Please see the introductory part of the following work  by: Stachel and Iftime.
The Lorentz group $\operatorname{SO}(3,1)$ in this picture is just a subgroup of $\operatorname{diff}(M)$. In the case when M is a Minkowski space, then the Lorentz group is a subgroup of the Poincaré group which the automorphism group of the Minkowski space.
Any bundle of tensor fields and forms on the base manifold has the above structure of a natural bundle, because diffeomorphisms can be canonically lifted to the fibers.  However , there are important bundles where there is no such a canonical lift. The most important example is the spinor bundle, where we do not have a canonical way to define a diffeomorphism (general coordinate transformation). We need to introduce vielbeins, i.e., sections of a frame bundle.  
The spinor bundle belongs to a family called gauge-natural bundles, in which the full diffeomorphism group cannot be lifted into a bundle automorphism, however a subgroup of which namely the isometry group can be lifted by means of what is known as the Kosmann lift. In particular, this lift defines a Lie derivative (i.e., a local diffeomorphism) of a spinor along a killing vector. Please see the following review  by  Fatibene, Ferraris, Francaviglia and Godina. Some elaboration of the Kosmann lift is given in this PSE question.
In the case of the Minkowski space with the Minkowski metric, the Kosmann lift allows lifting the action of the Lorentz group to spinor fields, since a Lorentz transformation is an isometry.
