How to multiply matrices (the physicists way) using index notation?

Question

N.B For this question we are only working in in flat cartesian space, not curved space-time.

Question: Consider $A$, $B$, $C$ and $D$ to be $n \times n$ matrices. Write down the following matrix multiplications using index notation and Einstein summation rules:

1. $A=B(C+D)$
2. $A=BCD$

According to my notes:

$$C=AB \qquad \text{means}\qquad C^i{\,_j}=A^i{\,_k}B^k{\,_j}\tag{1}$$ $$D=BA \qquad \text{means}\qquad D^i{\,_j}=B^i{\,_k}A^k{\,_j}=A^k{\,_j}B^i{\,_k}\tag{2}$$ $$E=A^TB \,\,\,\quad \text{means}\qquad E^i{\,_j}=A{_{\color{red}{k}}}^{\,\color{blue}{i}}B^k{\,_j}\tag{3}$$

For question 1. multiplying out gives $A =BC+BD$, and following the rules above I think the answer in tensor form should be $A^i{\,_j}=B^i{\,_k}C^k{\,_j}+B^i{\,_k}D^k{\,_j}$

For question 2. by my logic, $A=BCD$, so in tensor form this should be $A^i{\,_j}=B^i{\,_k}C^k{\,_\ell}D^\ell{\,_j}$

However, according to the authors' solutions, both of my answers are wrong:

$A=B(C+D)\implies A_{ij}=B_{ik}C^k{\,_j}+B_{ik}D^k{\,_j}=\delta^{k\ell}B_{ik}\left(C_{\ell j}+D_{\ell j}\right)$ $A=BCD \implies A_{ij}=B_{ik}C^k{\,_\ell}D^\ell{\,_j}=\delta^{km}\delta^{\ell n}B_{ik}C_{m\ell}D_{nj}$

---

So I followed the rules given by $(1)$, $(2)$, and $(3)$ and got the wrong answer. But, before I ask about the authors' solutions, there is something I don't understand regarding equation $(3)$.

According to a small section of my notes, I have that:

From the notes above, I think equation $(3)$ should be $$E^i{\,_j}=A{_{\color{red}{i}}}^{\,\color{blue}{k}}B^k{\,_j}\tag{4}$$ (where I have swapped the indices for the red and blue parts), since the transpose operation interchanges rows and columns and the section of notes shown above states that the lower index of the matrix labels the columns. So why is equation $(3)$ written that way when rows and columns are clearly being interchanged? By the way, I know that equation $(4)$ which I have written is actually wrong also, since the 'dummy index', $k$ is written up twice. But, what I am questioning here is what the transpose does to the indices.

From the image above, I have also been taught how to use the Kronecker delta metric (or identity matrix), which in the case of a $2\times 2$ identity matrix is $\delta^{ij}=\delta_{ij}=\begin{pmatrix}1 & 0 \\ 0 & 1 \\\end{pmatrix}$. So the way I see it, multiplying by the $\delta^{ij}$ essentially changes a row vector to a column vector, and my understanding of this is because the 'dummy index' ($j$ in the example in the top right of the image) is contracted upon so that the Kronecker is only non-zero when $i=j$ and this 'somehow' changes the original $U_i$ to $U^i$. Why this works the way it does is a question for another time, but for now, I would like to focus on trying to understand why the authors' solutions look the way they do for matrix multiplication.

For the authors' first answer, why are both indices for $A$ and $B$ written down ($A_{ij}$ and $B_{ik}$) when I was explicitly told to keep one index up and the other down?

$A=B(C+D)\implies A_{ij}=B_{ik}C^k{\,_j}+B_{ik}D^k{\,_j}=\delta^{k\ell}B_{ik}\left(C_{\ell j}+D_{\ell j}\right)$

For the last equality, I am guessing that the presence of $\delta^{k\ell}$ is used to contract the indices in the bracketed factor ($C$ and $D$), could someone please explain why this is done?

For the authors' second answer:

$A=BCD \implies A_{ij}=B_{ik}C^k{\,_\ell}D^\ell{\,_j}=\delta^{km}\delta^{\ell n}B_{ik}C_{m\ell}D_{nj}$

I simply have no idea what's happening on the far RHS of the above and I won't ask any questions about this yet as an answer to the first question may teach me how to understand this answer.

---

Final remarks

I purposely asked this on Physics-Stackexchange instead of Maths-Stackexchange as I am a student of physics and I don't want to be scared off by notation like $(\mathbf{v} \otimes \mathbf{w})_{ij} = v_i w_j$, which is the way I think a mathematician would write such an expression.

---

Update:

Answers so far have addressed nicely how the index notation can vary (both indices up/down or one up and one down). Now I just need to understand specifically what the $\delta_{ij}$ does to the indices of a matrix. This will help me to understand the authors' solution to 1. and 2.

Answer 1

You're right, equation (3) is wrong. When converting from implicit-index matrix notation to explicit-index notation, the contracted indices should always be the inner ones (the right index of the left factor and the left index of the right factor). As J. Murray said, it's the horizontal position of the index that determines whether it's a row or column index, not the vertical position.

Your equation (4) is correct except that it should be something like $δ_{k\ell}{A_i}^k{B^\ell}_j$, as I think you understand.

Your answers to questions 1 and 2 are also reasonable. They aren't wrong unless the question said that some of the matrices should have two lower indices.

The official answers are assuming, e.g., that $δ^{k\ell}B_{ik}={B_i}^\ell$. Strictly speaking, this is an abuse of notation. The two tensors named $B$ are different and should have different names. But it's common to abuse notation in this way, and when they do, you can safely assume that $δ$ raises or lowers an index without changing its horizontal position: that is, $δ^{k\ell}B_{ik} \ne {B^\ell}_i$. The order of the indices of $δ$ doesn't matter.

When evaluating $δ_{k\ell}{A_i}^k{B^\ell}_j$, you can either lower an index of $A$ to get $A_{i\ell}{B^\ell}_j$ or lower an index of $B$ to get ${A_i}^kB_{kj}$, and then sum over the repeated index. You can also just sum over both $k$ and $\ell$.

You can think of $U_i$ as a row vector and $U^i$ as a column vector, but beware that unless you consistently write matrices with ${{}^i}_j$ indices, products of these vectors with matrices won't always make sense. (Most obviously, you can't get a column vector from a row vector by multiplying it by $\begin{pmatrix}1&0\\0&1\end{pmatrix}$.)

Answer 2

The physicist's way to look at matrix indices is as book keeping devices that keep track of "rows" and "columns", thus, for an $n\times n$ matrix ${\bf A}$,

$$A_{ij}$$

is sufficient to define the entry in the $i^{\rm th}$ row and $j^{\rm th}$ column of ${\bf A}$. This entry is an essential part of ${\bf A}$, and totality of all $n\times n$ such $A_{ij}$ are ${\bf A}$.

For a "rank-2" tensor, ${\bf T}$, on a $n-$dimensional space? Not so much. Here we have four choices of indices:

$$ T_{ij},\ T_i^{\ j}, \ T^i_{\ j},\ T^{ij}$$

which each are elements of different geometric objects depending on whether ${\bf T}$ covariant, contra variant, or mixed. Moreover, they are not ${\bf T}$, they are representations of ${\bf T}$ in some basis. A transformation to a new basis yields completely different list of $T'_{ij}$, a different representation, but the geometric object being represented is the same.

One cannot say that about matrices because they are not geometric objects, they are lists of numbers, and their identity is that list. The list is not a representation of an object, it is the object. Since numbers are neither covariant or contra-variant, there should be no mixing of super/sub-scripts. It adds useless detail that contains no information.

Hence:

$$ ({\bf AB})_{ik} = A_{ij}B_{jk} $$

is enough to get it done with distributive and associative formulae.

Answer 3

However, according authors' solutions both of my answers are completely wrong.

In the absence of any further information, I would interpret "an $n\times n$ matrix" to mean "a $(1,1)$-tensor" whose first index is up and whose second index is down. Under that assumption, both of your answers are perfectly correct.

There exist objects with two downstairs (resp. upstairs) indices, and they are called $(0,2)$-tensors (resp. $(2,0)$-tensors). You can lay out their components in an $n\times n$ array just like a matrix, but their transformation properties are slightly different. For that reason, it's important to specify whether an object with two indices is a $(2,0)$-tensor, a $(1,1)$-tensor, or a $(0,2)$-tensor, but the language used in your question is conventionally understood to imply that $A,B,C,$ and $D$ are $(1,1)$-tensors.

[...] since the transpose operation interchanges rows and columns and the section of notes shown above states that the lower index of the matrix labels the columns.

Conventionally, it is the second (i.e right-most) index which labels the column, not the lower one. $A^i_{\ \ j}$ is the component of $A$ in the $i^{th}$ row and $j^{th}$ column; $B_{a}^{\ \ \ b}$ is the element of $B$ in the $a^{th}$ row and $b^{th}$ column.

---

I purposely asked this on Physics-Stackexchange instead of Maths-Stackexchange as I am a student of physics and I don't want to be scared off of notation like $(\mathbf{v} \otimes \mathbf{w})_{ij} = v_i w_j$, which is the way I think mathematician would write such an expression.

I'm not sure why that notation is particularly upsetting - $\mathbf v \otimes \mathbf w$ is the matrix (more specifically, the $(0,2)$-tensor) whose $(i,j)^{th}$ component is $v_i w_j$. In physics, the standard notation for such an object is $\mathbf v \mathbf w$; admittedly this has one fewer symbol in it, but in my experience the omission of symbols ultimately tends to cause more confusion for introductory students, not less. Your mileage may, of course, vary.