What is the covariant derivative in mathematician's language? In mathematics, we talk about tangent vectors and cotangent vectors on a manifold at each point, and vector fields and cotangent vector fields (also known as differential one-forms). When we talk about tensor fields, we mean differentiable sections of some tensor power of the tangent or cotangent bundle (or a combination).
There are various natural differentiation operations, such as the exterior derivative of anti-symmetric covariant tensor fields, or the Lie derivative of two vector fields. These have nice coordinate-free definitions.
In physics, there is talk of "covariant derivatives" of tensor fields, whose resulting objects are different kind of tensor fields.
I was wondering, what is the abstract interpretation of the general notion of a covariant derivative in terms of (tensor products of) tangent vectors and vector fields.
 A: The notion of covariant derivative is equivalent to the notion of connection. More precisely, for every connection $\nabla$ and vector field $X$, the operation $\nabla_X$ is a covariant derivative.
Connections $\nabla$ on the tangent bundle $TM$ of a manifold are usually induced by a metric, this is the so called Levi-Citiva connection. It is essentially determined by the requirement that the scalar product fulfills the product rule
$$ d_X\langle v,w\rangle = \langle \nabla_X v, w\rangle + \langle v, \nabla_X w \rangle .$$
Note that this does not yet determine the connection completely, one has to add the additional requirement that there is no torsion,
$$ \nabla_X Y - \nabla_Y X = [X,Y] .$$

This is a coordinate-free specification of a connection. Note, however, that unlike the Lie-derivative or the derivatives of differential forms, which are defined in terms of the manifold alone, connections represent additional data. Many different connections are possible on a single manifold, whereas the other two notions of derivative are unique.
A: A covariant derivative on some manifold is a map which takes each differentiable tensor field of type (n,m) to a tensor field of type (n,m+1) with the following properties:


*

*Linearity

*Leibnitz rule

*Commutativity with contraction

*Consistency with notion of tangent vectors


Additionally, one may wish to impose the condition of vanishing torsion.
Using abstract index notation the above conditions translate to the following conditions for the covariant derivative $\nabla_a$:


*

*$\nabla_a (A T^{a_1 \dots a_n}{}_{b_1 \dots b_m} + B \hat T^{a_1 \dots a_n}{}_{b_1 \dots b_m} ) = A \nabla_a T^{a_1 \dots a_n}{}_{b_1 \dots b_m} + B \nabla_a \hat T^{a_1 \dots a_n}{}_{b_1 \dots b_m}$

*$\nabla_a (T^{a_1 \dots a_n}{}_{b_1 \dots b_m}\hat T^{\hat a_1 \dots \hat a_n}{}_{\hat b_1 \dots \hat b_m}) = (\nabla_a T^{a_1 \dots a_n}{}_{b_1 \dots b_m}) \hat T^{\hat a_1 \dots \hat a_n}{}_{\hat b_1 \dots \hat b_m} + T^{a_1 \dots a_n}{}_{b_1 \dots b_m} \nabla_a \hat T^{\hat a_1 \dots \hat a_n}{}_{\hat b_1 \dots \hat b_m}$

*$\nabla_a \big(T^{a_1 \dots c \dots a_n}{}_{b_1 \dots c \dots b_m}\big) = \nabla_a T^{a_1 \dots c \dots a_n}{}_{b_1 \dots c \dots b_m}$

*For any tangent vector $v^a$ and any sufficiently smooth scalar function $f$ you have $v(t)=v^a\nabla_a f$


The condition of vanishing torsion requires the covariant derivative to commute on sufficiently smooth scalar functions $f$, $[\nabla_a,\nabla_b]f=0$.
If you take the commutator of the covariant derivative acting on vector fields you get a nice algebraic interpretation of torsion $T$ and curvature $R$:
$
[\nabla_a,\nabla_b]v^c = T^d{}_{ab}\nabla_d v^c + R^c{}_{dab} v^d
$
Thus, torsion coefficients are like structure constants in a Lie algebra and curvature resembles a central charge. 
In general relativity torsion is required to vanish. Together with the condition of metric compatibility, $\nabla_a g_{bc}=0$, where $g_{ab}$ is the metric, this defines uniquely your covariant derivative. The ensuing connection is then the Christoffel connection.
