Tensors are mathematical objects that are needed in physics to define certain quantities. I have a couple of questions regarding them that need to be clarified:
Are matrices and second rank tensors the same thing?
If the answer to 1 is yes, then can we think of a 3rd rank tensor as an ordered set of numbers in 3D lattice (just in the same way as we can think of a matrix as an ordered set of numbers in 2D lattice)?
A second-order tensor can be represented by a matrix, just as a first-order tensor can be represented by an array. But there is more to the tensor than just its arrangement of components; we also need to include how the array transforms upon a change of basis. So tensor is an n-dimensional array satisfying a particular transformation law.
So, yes, a third-order tensor can be represented as a 3-dimensional array of numbers -- in conjunction with an associated transformation law.
Matrices are often first introduced to students to represent linear transformations taking vectors from $\mathbb{R}^n$ and mapping them to vectors in $\mathbb{R}^m$. A given linear transformation may be represented by infinitely many different matrices depending on the basis vectors chosen for $\mathbb{R}^n$ and $\mathbb{R}^m$, and a well-defined transformation law allows one to rewrite the linear operation for each choice of basis vectors.
Second rank tensors are quite similar, but there is one important difference that comes up for applications in which non-Euclidean (non-flat) distance metrics are considered, such as general relativity. 2nd rank tensors may map not just $\mathbb{R}^n$ to $\mathbb{R}^m$, but may also map between the dual spaces of either $\mathbb{R}^n$ or $\mathbb{R}^m$. The transformation law for tensors is similar to the one first learned for linear operators, but allows for the added flexibility of allowing the tensor to switch between acting on dual spaces or not.
Note that for Euclidean distance metrics, the dual space and the original vector space are the same, so this distinction doesn't matter in that case.
Moreover, 2nd rank tensors can act not just as maps from one vector space to another. The operation of tensor "contraction" (a generalization of the dot product for vectors) allows 2nd rank tensors to act on other second rank tensors to produce a scalar. This contraction process is generalizable for higher dimensional tensors, allowing for contractions between tensors of varying ranks to produce products of varying ranks.
To echo another answer posted here, a 2nd rank tensor at any time can indeed be represented by a matrix, which simply means rows and columns of numbers on a page. What I'm trying to do is offer a distinction between matrices as they are first introduced to represent linear operators from vector spaces, and matrices that represent the slightly more flexible objects I've described
A matrix is a special case of a second rank tensor with 1 index up and 1 index down. It takes vectors to vectors, (by contracting the upper index of the vector with the lower index of the tensor), covectors to covectors (by contracting the lower index of the covector with the upper index of the tensor), and in general, it can take an m upper/n-lower tensor to either m-upper/n-lower by acting on one of the up indices, to m-upper/n-lower by acting on one of the lower indices, or to m-1-upper/n-1-lower by contracting with one upper and one lower indices.
There is no benefit to matrix notation if you know tensors, it's a special case where the operation of tensor product plus one contraction produces an object of the same type. The tensor notation generalizes the calculus of vectors and linear algebra properly to make the right mathematical objects.
Strictly speaking matrices and rank 2 tensors are not quite the same thing, but there is a close correspondence that works for most practical purposes that physicists encounter.
A matrix is a two dimensional array of numbers (or values from some field or ring). A 2-rank tensor is a linear map from two vector spaces, over some field such as the real numbers, to that field. If the vector spaces are finite dimensional then you can select a basis for each one and form a matrix of components. This correspondence between matrices and rank-2 tensors is one-to-one so you can regard them as the same thing, but strictly speaking they are just equivalent.
You can make up cases of infinite dimensional vector spaces where no meaningful representation in terms of matrices for the corresponding tensors is possible even when the field is the real numbers and the matrices can have an infinity of components. Some of these examples are relevant to physics, e.g. when the vector spaces are functionals whose dimension is (in lose terms) uncountably infinite. For this reason it is a good idea to bear in mind the distinction between what tensors and matrices of arrays actually are, even if you are just a physicist.
I know this is an old thread. But I think there still is a point missing, so if people still come to this post for reference, let me have a swing at it.
I want to argue from a mathematical standpoint that is somewhat geometric. We want to take vectors from some vector space $V$. We will especially not demand that $V$ be an $\mathbb{R}^n$ for some dimension $n$, but let's say for simplicity that $V$ is finite dimensional. There are two different kinds of operations on this vector space that are relevant in the usual tensor picture, one being $\begin{pmatrix}0\\2 \end{pmatrix}$ tensors - which are easily described with no technical clutter - and the other are linear maps described by $\begin{pmatrix}1\\1 \end{pmatrix}$ tensors - which require technical clutter, so we stay brief here. We will also need to talk about bases. So let's go in three steps.
Premature Postscript: This grew a bit out of proportion. I tried to elaborate because I know that my students often battle handwavy and brief answers that don't stretch the important cues.
A good source to get into the matter (and very readable) is Nadir Jeevanjee's Introduction to Tensors and Group Theory for Physicists.
This is it. That's really what a tensor is: it takes two vectors and spits out a real number. And it does so in a linear fashion.
This definition is independent of the vectors space $V$, tensors can be described on any vectors space. An example for the vector space $V$ could be the space of all the possible velocities a billiard ball could have (it is a vector space because you can stretch and add velocities, and there really isn't that much more for some set to qualify as a vector space). And a tensor? As mentioned above, any multiniear map will do, but something meaningful to physisc could be the "kinetic energy tensor" $$ T(v,w) = \frac{m}{2}v \cdot w,$$ whose diagonal is exactly $T(v,v) = E_{\textrm{kin}}(v)$.
Now, notice one thing: Never in this definition or the example have we mentioned anything about coordinates or $\mathbb{R}^3$. This is important. The tensor is an object that can exist in its full splendor, free and independent of any coordinate system. To a theoretician (or any physicist), this is a pleasing result: There is no experiment out there that can determine whether the coordinate system of the world is cartesian or polar or spherical. These are figments of the human mind. A good theory shouldn't start out depending on an arbitrary choice, it is better if tensors are well-defined entities before we get lost in coordinate systems. And that's what we did here.
But then again, our minds work pretty well in coordinate systems, or at least they were well trained to do. So what happens if we choose a basis? Then the vectors $v,w \in V$ can be decomposed into a sum over the set of basis vectors $\{e_i\}$ times respective scaling factors $\{v^i\}$ for each of them. With the sum convention: $$ v = v^i e_i, \qquad w = w^i e_i.$$ We plug it into the tensor definition and see what comes out. \begin{align} T(v,w) &= T(v^i e_i, w^j e_j)\\ &= v^i T(e_i, w^j e_j)\\ &= v^i T(e_i, e_j) w^j \\ &=: v^i T_{ij} w^j. \end{align} The first equality is just insertion (with proper care for the indices), the second equality is the linearity of the first argument ($v^i$ is just a real number, it can be pulled out), the third equality is the linearity in the second argument, and finally we introduce the definition $$T_{ij} := T(e_i,e_j).$$ This is a new quantity we defined by taking the tensor and applying it to all pairs of basis vectors. If the vector space has dimension $n$, then we get $n^2$ real numbers, while the components $v^i$ and $w^j$ each form $n$ real numbers.
And now one comes up with an entirely arbitrary way of storing all this information: matrices. A matrix in itself is nothing more than a table (they were, historically, even called tables for a while). Mere MS-Excel spreadsheets. But motivated by the equation that we just derived, people came up with the idea: hey, let's arrange the $v^i$ and $w^j$ into these rows and columns of numbers and let's arrange the numbers $T_{ij}$ into this nice square block of numbers. And to remember how to deal with them, let us introduce a way to multiply them with one another.
A matrix (including square matrices in $\mathbb{R}^{n \times n}$ as well as row matrices in $\mathbb{R}^{1\times n}$ and column matrices in $\mathbb{R}^{n \times 1}$, commonly referred to as vectors) is, as mentioned in another answer, nothing but a way to store information. The matrix multiplication rule ("row times column") is additional information on top of that. It's just a way to correctly handle the information stored in the matrix and the vectors, which is bare real numbers.
This is the sense in which we consider vectors to lie in some $\mathbb{R}^n$ and tensors to be matrices in $\mathbb{R}^{n \times n}$. Vectors actually lie in some $n$-dimensional vector space $V$, and tensors are bilinear maps that take two of these vectors and give a real number. However, after choosing a basis $\{e_i\} \subset V$, all the information we need to recover the full vector $v$ are its components $\{v^i\}$ in that given basis, and all we need to fully know a tensor $T$ are its values on the basis vectors $\{T_{ij}\} = \{T(e_i,e_j) \}$. This pushes the basis under the rug, but then weird stuff happens when one changes bases.
So why are most answers here centered around tensors being linear maps that take a vector in $\mathbb{R}^n$ to another vector in $\mathbb{R}^n$? Because of course there is a close similarity.
We look at the coordinate representation of the tensor multiplication again. There, we wrote $$T(v,w) = v^i T_{ij}w^j.$$ This bears close similarity to the coordinate representation of the inner product, $$v \cdot u = v^i u_i,$$ we just need to replace $$u_i = T_{ij}w^j = T(e_i,e_j)w^j = T(e_i,w^j e_j) = T(e_i,w)$$ in the equation.
In that sense, we find a new way of understanding the tensor. Instead of just taking two vectors and returning a real number, we can consider the tensor to take a vector $w$, linearly transform it into a new vector $u$, and then calculate the inner product between $v$ and $u$.
An aside: At this point, I left out a few subtleties, because to get the full picture, we would need to further talk about the inner product. This is because the inner product defines the metric tensor (or vice versa), which we still need if we want to get from the covariant components of the $u_i$ to the contravariant components $u^i = \eta^{ij} u_j$. We will not dwell on this here. I suppose that was already discussed elsewhere, anyway.
This is a pet peeve of mine. Having in the early part of my career been a geometer. Much of the discussion before is correct. A tensor of various ranks are linear transformations. However, a tensor is an invariant under coordinate systems selected.
Easiest way to think of it is a vector is a magnitude and direction and only can be expressed as an array once a coordinate system is chosen. Likewise a rank 2 tensor can only be expressed as a matrix when a coordinate system is chosen.
That is why it is used in physics like the stress energy tensor or the refractive index tensor of anistropic crystals. It's this coordinate invariance that makes it useful for describing physical properties.
No. A matrix can mean any number of things, a list of numbers, symbols or a name of a movie. But it can never be a tensor. Matrices can only be used as certain representations of tensors, but as such, they obscure all the geometric properties of tensors which are simply multilinear functions on vectors.
$$ \def\cR#1{\color{red} {#1}} \def\cG#1{\color{green}{#1}} $$
They seem so similar, but...
There is often confusion regarding indices when we transform a rank-2 tensor, $T_{ij}$. Because matrices can represent rank-2 tensors, it's tempting to just start multiplying. But index order is crucial.
Now, regular matrix multiplication, $C=AB$ sums over A's second index and B's first index: $C_{ab}=A_{a\cR{c}}B_{\cR{c}b}$, where $\cR{c}$ is the summation index ("dummy index").
If we use B's second index, $D_{ab}=A_{a\cR{c}}B_{b\cR{c}}$, then what we get is $D=AB^T$.
We read in a textbook that "T transforms like a tensor under rotation R" means that $T_{ab} \rightarrow T'_{ab} = R_{a\cR{c}} R_{b\cG{d}} T_{\cR{c}\cG{d}}$. Note crucially that one R operates on the first index, and the other R operates on the second index.
Therefore this does not mean $T \rightarrow T'=RRT$ in matrix notation. Wrong!
The correct matrix notation is $T \rightarrow T'=RTR^T$.
Seeing the correspondence may be tricky, getting lost in indices. Use the intermediary $D_{bc}=R_{b\cR{d}}T_{c\cR{d}}$ (analogous to above this is $D=RT^T$) so that $R_{a\cR{c}} R_{b\cG{d}} T_{\cR{c}\cG{d}}$ becomes $R_{a\cR{c}} D_{b\cR{c}}$. This also sums on D's second index so it equals $RD^T$. Substitute in the value of $D:RD^T=R(RT^T)^T=R(T^T)^T(R)^T=RTR^T. $
Example for 3: Matrix M (m11=x , m12=-y , m21=x^2 , m22=-y^2) .This matrix is not tensor rank 2. Test matrix M to rotation matrix.
