Connections in physics "are" the same as they are in mathematics but are usually interpreted as field potentials, with the exception of GR.

The interpretation follows naturally from the concept of a covariant derivative: local transformations of the field being studied must not change the physics involved (i.e. the Lagrangian must be invariant) so corresponding transformations of the connection cancel changes due to the field's transformation.

Take the case of quantum electrodynamics: The Lagrangian (density) is $$\mathcal{L}=\overline{\psi}\left(i\gamma^\mu\partial_\mu + m\right)\psi$$

You can check that it is invariant under the transformation $\psi\to e^{i\lambda}\psi$ when $\lambda$ is a constant. To make the transformation local we "promote" $\lambda$ to a function, but now we have an offending term $\overline{\psi}\gamma^\mu\partial_\mu\lambda e^{i\lambda}\psi$! All is well if we introduce the covariant derivative $\mathcal{D}_\mu=\partial_\mu-ie_0A_\mu$ such that $A_\mu\to A_\mu +\partial_\mu \lambda$ is the corresponding transformation.

The full Lagrangian is then $$\mathcal{L}=\overline{\psi}\left(i\gamma^\mu\partial_\mu + m\right)\psi + e_0\overline{\psi}\gamma^\mu A_\mu \psi + \frac{1}{4}F^{\mu\nu}F_{\mu\nu}$$

where $F_{\mu\nu}$ is the electromagnetic field strength tensor introduced to account for the dynamics of the potential (photon) field $A_\mu$

Connections in the context of relativity are instead the gravitational field strength since in our currently accepted theory gravity is not a "gauge field" like the photon field. Additionally, the identification of gravitation with spacetime curvature makes particles travel according to the geodesic equation which can recover the usual Gauss's Law for gravity in the Newtonian limit.

Connections in physics "are" the same as they are in mathematics but are usually interpreted as field potentials, with the exception of GR.

The interpretation follows naturally from the concept of a covariant derivative: local transformations of the field being studied must not change the physics involved (i.e. the Lagrangian must be invariant) so one introduces another "gauge" field which has dynamics of its own to cancel changes from the matter field's transformation.

Take the case of quantum electrodynamics: The Lagrangian (density) is $$\mathcal{L}=\overline{\psi}\left(i\gamma^\mu\partial_\mu + m\right)\psi$$

You can check that it is invariant under the transformation $\psi\to e^{i\lambda}\psi$ when $\lambda$ is a constant. To make the transformation local we "promote" $\lambda$ to a function, but now we have an offending term $\overline{\psi}\gamma^\mu\partial_\mu\lambda e^{i\lambda}\psi$! All is well if we introduce the covariant derivative $\mathcal{D}_\mu=\partial_\mu-ie_0A_\mu$ such that $A_\mu\to A_\mu +\partial_\mu \lambda$ is the corresponding transformation.

The full Lagrangian is then $$\mathcal{L}=\overline{\psi}\left(i\gamma^\mu\partial_\mu + m\right)\psi + e_0\overline{\psi}\gamma^\mu A_\mu \psi + \frac{1}{4}F^{\mu\nu}F_{\mu\nu}$$

where $F_{\mu\nu}$ is the electromagnetic field strength tensor introduced to account for the dynamics of the potential (photon) field $A_\mu$

Connections in the context of relativity are instead the gravitational field strength since in our currently accepted theory gravity is not a "gauge field" like the photon field. The identification of gravitation with spacetime curvature makes particles travel according to the geodesic equation which can recover the usual Gauss's Law for gravity in the Newtonian limit.