The notion of vector (not just in the sense of a point in a vector space) can be and is generalized in countless different ways. Here I just mention one that I personally find very intriguing, especially in relation to electromagnetism, and give some personal thoughts. References are given throughout the answer and at the end.
I think I heard of theories that use differential manifolds (eg to model spacetime) with a bundle of affine spaces, rather than vector spaces. Curtis & Miller mention such bundles in chap. 16:
Personally I see affine space more like a particular case of a differential manifold: one in which there is a flat affine connection. From this viewpoint the notion of vector on an affine space is just the same as that on a differential manifold (tangent vector), with the only difference that there's a "friendlier" (but less interesting) parallel transport in affine space.
At the same time, there's also an alternative way of seeing vectors (and their generalizations discussed below) in an affine space. It appears in the geometric calculus of Grassmann and Peano:
In geometric calculus, vectors aren't introduced as translations (even though they can also be used that way there), but as sort of points at infinity. This leads to a framework in which one works simultaneously on affine space, projective space, and (multi)vector space, yet keeping them somewhat distinct. I think it has great pedagogic potential for introducing and teaching these kinds of space.
There are a couple of brilliant and clear articles by Goldman that explain this "dialogue" between these three spaces in geometric calculus (which can only be glimpsed in Peano, although it's there on closer inspection). Goldman uses it for computer graphics:
I think it would be great if this viewpoint could be brought to differential geometry, but don't know of any works that have done that.
Ideas and objects extremely close to those of geometric calculus can also be constructed on a vector space, however, and from there they're naturally brought to differential geometry. This is an old and well-developed framework, albeit not yet as widely known as it deserves to be. The basic idea is very intuitive, and it's the same as in the geometric calculus:
A vector is usually identified by: (1) a direction, in the sense of a specific straight line; (2) an orientation on that line, with two possible choices; (3) a magnitude on that line, in the sense of a chosen segment there. All these three characteristics can be generalized, in an ambient space of dimension $D$:
Instead of a line we can take any $k$-dimensional flat subset with $k\le D$ (plane, hyperplane, etc.).
The orientation can be chosen on the $k$-dimensional flat subset itself (two choices), or on its complement, that is, the equivalence class of all subsets parallel to the specific subset. Figuratively, this corresponds to taking the orientation "around" the subset, rather than on it.
Also the magnitude ($k$-volume) can be taken on the flat subset, or on its complement. (This notion of magnitude doesn't require a metric; e.g. see Affine and convex spaces: blending the analytic and geometric viewpoints for an explanation.)
The freedom in these three choices, which can be combined, leads to "generalized vectors" that also have intuitive visual representations in 2D and 3D. Here are the drawings from Schouten's book, for the case of a 3D space:
The first kind of generalization leads to multivectors, which represent oriented areas, volumes, and so on (in differential geometry, tangent areas, volumes, and so on). The second kind of generalization leads to twisted vectors, which can represent notions such as rotational momentum, or the direction of flux through areas, and so on. The third kind of generalization leads to (multi)covectors (or forms in differential geometry), objects that can acts as basic measurement operations, or, in differential geometry, meant to be integrated.
So what we obtain are none other than the multivectors and multicovectors (that is, elements of the dual space) of linear algebra, with so called "twisted" or "straight" orientations, or "odd" and "even" as de Rham called them. They also correspond to completely antisymmetric, fully contravariant or fully covariant tensors. And they also correspond to the objects treated by the geometric calculus of Grassmann and Peano.
These objects represent very well many physical quantities, such as densities, forces, and especially the objects in electromagnetism. They fully express the symmetries and operational meaning of those objects.
Just two examples:
– Charge density is something that's supposed to be integrated in a region of space, to give the total charge in that region. And such integration must not depend on distances or metric, since it's a purely topological notion: we choose a closed surface in space, and we say what's the total charge in there. It turns out that this notion and operational meaning can be fully expressed by one of the objects above: a twisted 3-covector.
– The electric field corresponds to a straight 1-covector, and again this stems from its operational meaning: integrated on a curve (again, this is all topological), it yields a potential difference.
...And many other physical notions: all of electromagnetism and most part of mechanics. The most fascinating aspect is that these notions allow you to interpret the electromagnetic (Faraday) field as the motion of a continuum of strings (1D objects) in spacetime, just like classical mechanics treats of a continuum of particles (0D objects).
One object that still has some mystery from this point of view is the stress-energy-momentum tensor.
I recommend the works of Bossavit and of Hehl & al. below to see what the various electromagnetic fields correspond to.
There is a very extensive literature on these notions and their applications. Here are some further references.
To get acquainted with them and their use in electromagnetism:
For their geometric properties and application to physics in general:
Some remarks about the "natural" multivector form of the stress-energy-momentum tensor are given here:
This is just the tip of the iceberg, and it's a huge iceberg. Some forward- and backward-searches from these references will reveal many many other interesting ones.