# Converting between matrix multiplication and tensor contraction

If I have $A^{\alpha \beta} B_{\beta \gamma}$ then this should be the equivalent to the following matrix multiplication: $AB$ since we're summing over the columns of $A$ and the rows of $B$. By the same logic, if I have $A^{\alpha \beta} B_{\gamma \alpha}$ then this is $BA$ in matrix notation. But what if I was doing $A^{\alpha \beta} B_{\alpha \gamma}$? Would that be equivalent to $A^{T}B$ or $B^T A$ in matrix notation or something else entirely? Furthermore, what would my index notation look like for the resultant? Would it look something like $C^{\beta}_{\gamma},$ or would one index be further to the left than the other?

• Tensors are not matrices. IMHO we should not try to convert between these objects. That being said, only $(1,1)$ tensors behave like matrices, and therefore $A^a{}_b B^b{}_c$ can be written as $AB$. Any other product is just abuse of notation. – AccidentalFourierTransform Apr 27 '16 at 20:32
• Why would a (2,0) tensor multiplied by a (0,2) tensor as in the first example not multiply as a matrix? If these are not to be looked at as matrices, what is the most practical way to do actual computations on the components of these kind of things? Do you just crank through each index by hand? Also, how do you represent these things for your final answer? If not as a matrix, the only way I could think about doing this would be to literally list each component of the resultant line by line, but that seems kind of ugly. – Dargscisyhp Apr 27 '16 at 20:36
• @Dargscisyhp Just out of curiosity, if you were to define it as a matrix multiplication, how would you do a computation besides to crank through the indexing by hand? Isn't every actual computation of a matrix multiplication where we do the whole "row by column" thing over and over just cranking through the indexes? – tpg2114 Apr 27 '16 at 20:45
• @tgp2114 Sure, but looking at it in rows and columns puts it in a kind of "mnemonical" way that's very quick to do. To do by hand, it seems much easier to do it using matrix notation, if at all possible. Otherwise the only thing I can think of is, find the 00 component of tensor 1 from a list, find and multiply by 00 component of tensor 2 and then find and add to the product of the 01 component of tensor 1 and 10 component of tensor 2, and so on. I take your point, but from a practical point of view matrix multiplication just seems easier. Perhaps it's because I don't yet have practice. – Dargscisyhp Apr 27 '16 at 20:51
• it just depends if you are standing on top of your head that day. The only difference between the two is that the matrix is transposed. $A^TB = (AB^T)^T$ – Bill Moore Oct 25 '19 at 16:30

Slogan: Matrices are a tool to compute sums; tensors tell you which sums make sense.

When you convert between rank-2 tensors and matrices, the decision as to which index of the tensor labels the rows and which one labels the columns is purely conventional. Matrix multiplication is no more than a convenient way to write products of the form

$$K(i,k) = \sum\nolimits_j M(i,j)N(j,k),$$

where I have conciously refrained from using indices to label matrix elements; instead, the first argument labels rows and the second labels columns.

Imagine, for example, that you want to compute the contraction $A^{ij} B_{jk}$. Define the matrix $M$ by $M(i,j) = A^{ij}$, the matrix $N$ by $N(j,k) = B_{jk}$ and the matrix $K$ by $K(i,k) = A^{ij}B_{jk}$. Then match the definitions of tensor contraction* and matrix multiplication to see that $$K = MN.$$ Do the multiplication and read off the components of the contraction using the definition of $K$.

Now define $\tilde M(j,i) = A^{ij}$, $\tilde N(j,k) = B_{ij}$, $\tilde K(k,i) = A^{ij}B_{jk}$. The components of the contraction you want and the tensors you know are still all contained in $\tilde K$, $\tilde M$ and $\tilde N$, but differently. Matching up the definitions of multiplication and contraction (and transpose) once again now gives $$\tilde K^T = \tilde M^T\tilde N\quad\text{or, equivalently,}\quad\tilde K = \tilde N^T\tilde M.$$

Look at all this again. You computed the same tensor contraction, twice, using different products of different matrices. (Of course, in fact $\tilde M = M^T$, $\tilde N = N$ and $\tilde K = K^T$.) In reality you would conserve letters, so my $M$ would also be called $A$, but this is nothing more than an abuse of notation. Doing the sums is all matrix products are good for. The meaning of the sums has to be obtained from elsewhere, usually from the tensor contractions they represent. A “matrix” is not a geometric object (and thus can’t be a physical object either).

On the other hand, a tensor is a geometric object. I will not try to describe what tensors really are (unless you ask for it in the comments), but as an appetizer, try to contemplate the difference between

• vectors $u, v\in V$ and scalar-valued linear functions $\phi, \psi : V\to\mathbf R$, given that $\phi(u)$ and $\psi(v)$ make sense but $u(v)$ and $\phi(\psi)$ don’t; now remember that things like $\phi_iu^i$ are well-defined but those like $u^iv^i$ are not;
• linear maps $A, B : V\to V$ such as rotations and bilinear functions $f, g : V\times V\to\mathbf R$, given that $A(u)$ is a vector, $f(u,v)$ is a scalar, $A\circ B$ is a linear map, but $A(u,v)$ and $f\circ g$ are meaningless; now remember you can contract $A^i_ju^j$, $f_{ij}u^iv^j$ and $A^i_jB^j_k$ (and look at the arrangement of free indices) but neither $A^i_ju^iv^j$ nor $f_{ij}g_{jk}$.

* I expect that your tensors and their contractions are defined in the physicicts’ tradition as collections of coordinates with incomprehensible properties involving large sums. This definition is decidedly not the most easily understood one; try searching or asking on MathOverflow for the one using universal properties.

As @AccidentalFourierTransform said, it's a mistake to try and think of tensors as matrices, as you rapidly end up going mad when you need to compute $A^{\alpha\beta\gamma\delta}B_{\beta\delta}$ or something. That said $A^{\alpha\beta}B_{\alpha\gamma}$ 'looks like' $A^TB$.

The right way to think of tensors is as multilinear functions, because that is what they are, in fact. In the index notation each index then corresponds to a slot in the function. So a $(2,2)$ tensor $T$ is a function $T(\_,\_;\_,\_)$ where the first two arguments want one-forms and the second two vectors. This then tells you how to write the indices of the components: not above each other, but in the order of the arguments, so $T^{\alpha\beta}{}_{\gamma\delta}$, and not $T^{\alpha\beta}_{\gamma\delta}$, which is ambiguous as to argument order.

It is really important to remember that tensors are multilinear functions, because if you don't do that you will inevitably fall into the terrible pit of index-gymnastics and become impaled on the poisoned spikes of contravariance and covariance, from which only a few ever escape. Remember there is geometry, not just indices and transformation rules.

One way of avoiding this trap while still being able to work in a convenient notation is to use the Penrose abstract-index notation. The most convenient thing of all is that you don't have to change anything to use it, though you might want to change the symbols you pick indices from.

• +1 I'd like to point out that, in practice, nobody ever computes $A^{\alpha\beta\gamma\delta}B_{\beta\delta}$ by hand: we use computers. I've read somewhere that tensor contractions was one of the motivations for Turing's work. Leibnitz had a mechanical device that simplified tensorial expressions. There are hieroglyphics that depict Egyptians going crazy trying to calculate the components of the Riemann tensor. History of science, people :-) – AccidentalFourierTransform Apr 27 '16 at 21:04
• It's a bit disconcerting for me to hear that it's improper to represent these things as matrices, as that is what the text that I'm using does (Carroll's Spacetime and Geometry). For purposes of multiplying or adding components of tensors, why is it improper to look at a (2,0) or (0,2) tensor as a matrix? It's a bit difficult for me to conceptualize this as geometry yet. I'm hoping that understanding will come by working through the problems. I also wish there was a solution's manual to this book, as this is self-study. – Dargscisyhp Apr 27 '16 at 21:21
• @AccidentalFourierTransform: I really meant that that contraction just makes no sense as matrix multiplication at all. You are right about the computer stuff: one of the first algebra systems, LAM was used to compute the curvature of (I think) the LTM metric. This had previously been done by a PhD student (I think it was their PhD) in a few months. LAM did it in a few minutes, and found mistakes in the manual calculation (some details here may be wrong: I was told it 25 years ago by the person who wrote LAM 25 year before that). – user107153 Apr 27 '16 at 21:54
• @Dargscisyhp: If you really want to regard rank-2 tensors as matrices, don’t let the traditional linear algebra notation confuse you, as it’s actually the (1,1) tensors for which it works best: e.g. you can map a matrix product $AB\cdots C$ to a contraction $A^i_jB^j_k\cdots C^k_\ell$. You can’t contract (0,2) tensors this way: in a sense, they should be then considered as “matrices with columns and columns”. (You see how quickly it gets confusing.) – Alex Shpilkin Apr 27 '16 at 22:27
• @Dargscisyhp: the practical answer is that you can't represent tensors as matrices in any easy way: you can get away with doing it for $(2,0)$, $(0,2)$ and $(1,1)$ tensors, but what do you do with an $(n,m)$ tensor?. More importantly, a vector isn't a bunch of numbers, it's a little arrow which has a representation as a bunch of numbers in some basis, but it's not the numbers: it's a geometrical object. So, therefore, is a tensor. MTW is 46 years old now, and physicists need to start reading it! (Sorry, this is perhaps contentious and anyway off-topic for SE: I'll stop.) – user107153 Apr 28 '16 at 11:20

As a mathematician, I would like to point out the following technicality:

What we (can) represent as a matrix are tensors with one index up and one index down, that is elements of $V\otimes V^*$ (where $V$ is a vector space, or a vector bundle over a manifold). A tensor with two indexes up is a bilinear form on $V$, and one with two indices down a bilinear form on $V^*$, and therefore there is no (natural) way of contracting, and that's where the metric comes into play: you use it to convert a bilinear form into a map $V\to V$ (i.e. an element of $V\otimes V^*$) or $V^*\to V^*$. Symplectic forms can have similar roles in various situations.

Of course, if you have two tensors $A^\mu_\nu,B^\eta_\theta$, which can be represented as matrices $A,B$ when you fix a basis of your vector space, then the contraction $A^\mu_iB^i_\nu$ is represented by the matrix multiplication $AB$.