Differentiating between tensors of different ranks In my course on tensors matrices have been given as an example of a 2nd rank tenor, as they involve two indices, and similarly a vector as a 1st rank tensor. As it is possible to have a vector space of matrices I struggle to see how these examples are consistent.
Question: How is the rank of a tensor defined? Is a matrix a second rank tensor? Is a vector a first rank rank tensor? Can a matrix be considered an element of a vector space? If yes to all these then is a matrix a first or second rank tensor and why?
Any help or resources to point me in the right direction would be greatly appreciated.
 A: You're conflating two definitions of vector, an element of a vector space (satisfying some famous axioms) and a quantity that transforms as a rank-$1$ tensor. A matrix is an element of a suitable vector space of matrices, but to every tensor rank there is a vector space of applicable vectors, and that includes rank-$2$ matrices. (We can subdivide these into type $(p,\,q)$ with $p=2,\,q=0$ vs. $p=q=1$ vs $p=0,\,q=2$, but that's another story.)
A: The definition you need to know is that of "an $(r,s)$ tensor over a vector space $V$". Just speaking of a tensor of some rank is not a precise enough terminology to clarify your misunderstanding (which clearly stems from not having clear definitions). So, here it is:

Definition.
Let $V$ be a finite-dimensional vector space over $\Bbb{R}$, and let $r,s\geq 0$ be integers. An $(r,s)$ tensor over $V$ is defined to be a multilinear mapping $f:\underbrace{V^*\times\cdots \times V^*}_{\text{$r$ times}}\times
\underbrace{V\times\cdots \times V}_{\text{$s$ times}}\to\Bbb{R}$. The space of all such tensors is denoted $T^r_s(V)$.

For example, a $(0,0)$ tensor on any vector space $V$ is just an element of $\Bbb{R}$ (a scalar). A $(1,0)$ tensor is a linear mapping $V^*\to \Bbb{R}$, i.e an element of $V^{**}$, which from linear algebra is canonically isomorphic to $V$. So, a $(1,0)$ tensor on $V$ amounts to the same thing as an element of $V$, i.e a vector in $V$. A $(0,1)$ tensor over $V$ is a linear mapping $V\to\Bbb{R}$, i.e an element of $V^*$. We refer to both $(0,1)$ and $(1,0)$ tensors on $V$ as rank one tensors.
A $(0,2),(1,1),(2,0)$ tensor over $V$ are all referred to as second rank tensors. The most common example of a $(0,2)$ tensor on a vector space $V$ is an inner product $\langle\cdot,\cdot\rangle:V\times V\to\Bbb{R}$; this is an important tensor because it is what gives meaning to geometry on a vector space (lengths, angles). Next, one can show that the space $T^1_1(V)$ of $(1,1)$ tensors (i.e bilinear maps $V^*\times V\to \Bbb{R}$) is naturally isomorphic to $\text{End}(V)$ (the space of linear mappings $V\to V$). In symbols, $T^1_1(V)\cong \text{End}(V)$ is a canonical isomorphism. Finally, by choosing a basis for $V$, we can provide an explicit isomorphism $\text{End}(V)\cong M_{n\times n}(\Bbb{R})$, where $n=\dim V$. In other words, every $(1,1)$ tensor on $V$ can be uniquely identified with an endomorphism on $V$, and furthermore by a choice of basis, this can be identified with a matrix.
So, a matrix is NOT the same thing as a $(1,1)$ tensor, but by a particular choice of basis, every $(1,1)$ tensor gives rise to a matrix, and conversely.
I should mention that once we choose a basis for $V$, we can construct isomorphisms $T^1_1(V)\cong T^2_0(V)\cong T^0_2(V)\cong \text{End}(V)\cong M_{n\times n}(\Bbb{R})$, since all these vector spaces have the same dimension of $n^2$, but the only natural isomorphism is $T^1_1(V)\cong \text{End}(V)$. So, bottom line: it is bad to think of matrices as (rank 2) tensors, unless you know exactly what you're talking about (in which case you wouldn't be confused).
Very often in introductory courses one usually considers $V=\Bbb{R}^3$, and works only with the standard basis $\{e_1,\dots, e_n\}$, so one doesn't really care about all this (frankly elementary) linear algebra, and essentially considers tensors as "certain arrays of numbers" (and hence the notion that a matrix is a second rank tensor), but this is a very dangerous way of thinking of things.
Slightly more careful presentations of tensors may talk about certain collections of numbers satisfying a certain transformation law under the general linear group. This definition is equivalent to the definition I gave above using multilinearity once we note that given any basis $\{e_1,\dots, e_n\}$ of $V$ with dual basis $\{\epsilon^1,\dots, \epsilon^n\}$, we can define the components $T^{i_1\dots, i_r}_{\qquad j_1\dots, j_s}$, and conversely, given these components, we can get back a multilinear map.
Note that the usual definition of vectors/tensors as encountered in physics texts is also a special case of the above definition. In physics, one is often interested in a smooth manifold $M$ (it can be the configuration manifold of classical Lagrangian mechanics, or the symplectic manifold of Hamiltonian mechanics, or the spacetime manifold as considered in special/general relativity). To each point $p\in M$, we can consider the tangent vector space $V=T_pM$. We can then consider tensors on this particular vector space. So, the usual definition can be applied here (and typically in physics, we like to consider several tangent spaces simultaneously; so a choice of a tensor at each tangent space yields a tensor field).

I should also remark that the more algebraic definition of an $(r,s)$ tensor on $V$ is as an element of the tensor product $V^{\otimes r}\otimes (V^*)^{\otimes s}$, where this space is defined using the universal property (but this is definitely too much abstraction for a first introduction, so you can ignore this).
A: $\mathbb{R}^n$ is a vector space, but unlike ordinary vector spaces it comes with a natural basis $1_* =(1_i)$. Thus it not only belongs to the category of vector soaces, $Vect$, it belongs to $BsVect$, the category of vector spaces equipped with a basis.
Now, when we fix a vector space $V \in Vect$ we can choose a vector $v \in V$ and we can obtain its coodinate vector by choosing a coordinate system. But which one are we to choose? There is no natural coordinate system, so we choose them all and then there will be a transformation law linking these different representations of the same vector. This is the coordinate tensor transformation law.
But what if we choose instead $V \in BsVect$? Well, here the space has a natural basis, the one it is equipped with and so we can dispense with the transformation law and simply identify a vector with its representation in the basis it is equipped with. So here, a vector as a coordinate tensor does not require a transformation law.
Thus vectors are represented differently as coordinate tensors if they belong to $Vect$ or to $BsVect$. The former requires a transformation law, and the latter doesn't.
Now if we choose a coordinate vector in $\mathbb{R}^n$, then this does not represent a vector in $Vect$ as it is not equipped with a transformation law. So here it is not a coordinate tensor. However, it does represent a coordinate tensor in $BsVect$ as here it does not require a transformation law.
Similarly, for a matrix.
As for your questions:


*

*How is the rank of a tensor defined?


A tensor, when written in components, will have both upper and lower indices. Let $p,q$ be the number of them respectively. Then it's type is $(p,q)$ and its rank is $p+q$. We write the category of all tensors of type $(p,q)$ on the vector space $V$ as $\otimes^p_q V$



*Is a matrix a second rank tensor?


It is not a coordinate tensor when concieved as representing a tensor in $\otimes^p_q V$ when the underlying space $V$ is not equipped with a basis. It is, when it does and since a matrix $A^i_j$ has a single upper index and a single lower index, then it  is a tensor of type $(1,1)$ and so has rank 2.



*Is a vector a first rank tensor?


Yes, always. A vector $v$, written as a coordinate tensor has the form $v^i$. Thus it has type $(1,0)$ and so rank 1.



*Can a matrix be considered an element of a vector space?


Yes.



*If yes to all these then is a matrix a first or second rank tensor and why?


Consider a $3 \times 3$ matrix, it can be alternatively considered as a column vector of length 9.
Now, this matrix, in either of its forms cannot be a coordinate tensor when the underlying space is not equipped with a basis. It does when the space is so equipped. And in particular, since $\mathbb{R}^n$ is so naturally equipped, then here a matrix is a tensor in either of the two forms listed above. In the first form, it is a coordinate tensor of rank 2 and in the second, a tensor of rank 1.
