What is a tensor? has been asked before, with the most highly up-voted answer defining a tensor of rank $k$ as a vector of a tensor of rank $k-1$.
But if a scalar is defined as a physical quantity with a magnitude, a vector as a physical quantity with both a magnitude and direction, can a tensor be defined in a similar way as a vector with additional qualities?
I think a safer way, at base level, of thinking of a tensor, is to think of it as a function that takes in $n$ vectors, and spits out a number, and in each of it's arguments, satisfies:
$$T(a{\vec v} + b {\vec w}) = aT({\vec v})+ bT({\vec w})$$
where $a,b$ are numbers, and ${\vec v}, {\vec w}$ are vectors, and the other arguments of $T$ are suppressed. There is certainly additional detail to get into (about one-forms, gradients, and coordinate transformation), but this is the base idea.
Mathematicians model ideas and study their properties. In mathematics, a vector space models a set of things that behave like like little arrows when added together and multiplied by numbers.
The mathematical definition of a vector space is a set with an addition operation and a multiplication by number operation that follows 8 rules. Anything that meets the definition is of interest to mathematicians. For example the set of continuous functions on the interval [0,1] is a vector space.
Mathematicians discover things like every vector space has at least one basis. All bases of a given vector space have the same number of vectors. This number is called the dimension.
Since a tensor space meets the definition of a vector space, it is a vector space.
Physicists often find that mathematicians invent useful tools. But physicists are interested in modeling the behavior of the universe. They use the tools in different ways than mathematicians do. For example, they are often less interested in mathematical rigor.
For the most part, physicists use vectors to model things like space or momentum. They find vectors are useful if they have a norm (or metric or length) and all the components are the same kind of quantity. The forward direction is space. Sideways is space.
If all the components are the same, you can change the basis and still use the vector space to model the universe. If I am facing forward and you are at 45 degrees to the right, we both can use $F = ma$.
Because of this physicists are very interested in how vectors transform when you change the basis.
And this is where tensors differ. Tensors of rank 2 transform differently than tensors of rank 1. So to physicists they are different objects.
As an aside, note that in relativity, one of the components is different than the others. Physicists have defined a useful, not-quite-kosher metric. They have found that when you change the basis, you get a useful model of the universe from the point of view of an observer at a different velocity.
Another aside, mathematicians would consider the phase space of statistical mechanics to be a vector space. But physicists don't much care about the vector properties of phase space. They don't add vectors or change the basis. They just follow the trajectory of a point as a system evolves.
If you want antisymmetric tensors, there are well known geometric building blocks. For instance if you take two orthogonal vectors you can multiply them to get the oriented plane they span (with the orientation determined by the order you multiplied them). Similarly for three mutually orthogonal vectors, note there are 6 ways to multiply them, but because they pairwise anticommute, there are only two orientations. In an $n$-dimensional space you might have as many as $n$ mutually orthogonal vectors so have a rank $n$ object.
Now if you want to consider adding these higher rank antisymmetric things you can insist that addition distribute across multiplication. And now you need to make a choice, either assume all vectors anticommute, (not just the orthogonal ones) and use $\wedge$ for the multiplication symbol and get a grassmann algebra. Or you assume that a vector multiplied by itself gives it's squared length and you get a clifford algebra.
But actually, I lied about having to make a choice since we used the $\wedge$ to denote the antisymmetric product on vectors, we can have both products sinc ethey are defined on the same thing (linear combinations of products of orthogonal vectors). So we can have the grassmann algebra product and the clifford algebra product, and call it a geometric algebra. The basic building blocks are the products of orthogonal vectors, which are little oriented 1-volumes, 2-volumes, 3-volumes, ... or $n$-volumes. Everything else is a linear combination of those.
However, tensors do include symmetric tensors, so this doesn't give you those tensors as geometric objects. So you might have to think of the full space of tensors as multilinear functions on vectors.
When introducing the stress tensor to undergraduate engineers, I note that, if a body is deformed by a stress state, and we imagine it sliced by a plane, then there is a (force) vector acting across (but not normal to) the plane. To fully specify the stress state, you need to cut the body with three planes, each of which can be specified by its normal vector. So, stress is something that has a component vector associated with each direction in space. This 'something' is a (2nd order) tensor.
In short:
...a vector has a component scalar (for want of a better word) in each direction in space
...a (2nd order) tensor has a component vector associated with each direction in space
Alternatively:
...a vector has magnitude and direction
...a tensor has magnitude(s) and two directions (one for the component vector and one for the plane).
I find this gives a better physical picture than the mathematicians' 'something that transforms according to...' or 'a mapping between...'. I do that later.
Is there a physical interpretation of a tensor
From a geometric perspective, and keeping in mind the duality between (1, 2, 3...) -forms and (1, 2, 3 ...) -vectors, observe that a one-form (rank 1 co-variant tensor) can be pictured as a series of surfaces that produces a number when contracted with a vector; the number being the number of surfaces pierced by the vector.
Generalizing to higher ranks, a two-form can be pictured as a 'honeycomb' like structure that produces a number when contracted with a surface with a sense of circulation; the number being the number of tubes piercing the surface.
A three-form can be pictured as an 'egg-crate' like structure that produces a number when contracted with a volume; the number being the number of cells within the volume.
This is essentially laid out on pages 115 - 117 of MTWs "Gravitation". For example:
Tensors are a class of objects, which includes both scalars and vectors.
There are tensors of rank $n$ that are objects with both a magnitude and $n$ orthogonal directions, consisting of $3^n$ components. Examples of this includes the vector itself, the Dyad (made up of vectors) or the Triad (made up of Dyads), the tetrad...
Physically, a tensor is just a collection of vectors that physicists study as a single object because the vectors are related to one another in a physically "interesting" way.
Tensors don't possess additional properties to the ideas of magnitude and orientation of a vector; all that can be done is to assign a vector to certain arrangements of vectors of certain tensors.
