There are in fact several additional interpretations of tangent vectors. An equivalent definition of a tangent vector involves equivalence classes of curves.equivalence classes of curves.
The appropriate interpretation largely depends on the context of a problem. As QMechanic stated, we may take a linear algebra viewpoint, or view them in terms of differential geometry.
In the latter case, one has the bundle $TM \to^\pi M$ and sections are maps $s : M \to TM$ such that for the projection, $\pi \circ s = \mathrm{id}_M$. Thus a tangent vector is a section of $TM$. It can be viewed as,
$$TM = \bigcup_{p \in M}T_pM$$
that is, the union of all tangent spaces on the manifold. A tangent vector at a point $p$ lies in $T_pM$ which is a fibre of the bundle $TM \to^\pi M$.
This interpretation is useful in general relativity, wherein physics is cast in the language of differential geometry. On the other hand, if say, I were solving a problem in classical mechanics involving firing a cannonball, I would think of its velocity vector as simply some $v\in \mathbb R^3$.