Covariant form: Determining the order of the product of tensors Suppose ${U^\mu}_\nu$ and ${{V^\mu}_\nu}$ are matrices.
We know that in general $U^TVU \neq U V U^T$. Whenever I am reading products of the form
$${B^\gamma}_\mu = {U_{\mu}}^\sigma{{V^\nu}_\sigma}{U^\gamma}_\nu,$$
how do I quickly see in which directions the multiplications were carried out, i.e. if this is $B=U^TVU $ or $B= U V U^T$?
 A: The paired, contracted indices tell you what multiplies with what and when. It does not matter which order the contractions are carried out in, just as matrix multiplication is associative. Try putting summation signs in (i.e. skip the Einstein convention for a moment) and bracket each summation with its pair of indices to see whether this clears matters up for you. The distributivity of addition over multiplication and the commutativity of both operations guarantees that the order of contraction won't matter.
In fact, matrix multiplication is simply another way of specifying contraction:
the rows of a matrix $A$ contract with the columns of another matrix $B$ after-multiplying the first one to yield $A\,B$
If the components are $A_{i\,j}$ and $B_{j\,k}$ then $A\,B$ has components $\sum\limits_jA_{i\,j}\,B_{j\,k}$. So this is the rule you apply if you want, for example, to use matrix algebra to multiply your tensors. Let's assume the tensor ${U^\gamma}_\nu$ is written as a matrix $U$ with the upper index $\gamma$ numbering the rows. Likewise for ${V^\nu}_\sigma$: we write this as the matrix $V$ with the $\nu$ index standing for the rows.
Then, by the rule of matrix multiplication that rows contract with the columns of an after-multiplier, the matrix $U\,V$ holds the components of ${{V^\nu}_\sigma}{U^\gamma}_\nu$. The rows of the product $U\,V$ correspond to the index $\gamma$ in ${{V^\nu}_\sigma}{U^\gamma}_\nu$. The order is switched because the $\nu$ index of $U$ names the $\nu^{th}$ column of the $\gamma^{th}$ row of $U$, and this must align with the $\nu^{th}$ row of the $\sigma^{th}$ column in $V$. You'll need to write this out with, say, a $2\times2$ example to see this in detail.
Now we come to the entity ${U_{\mu}}^\sigma$. Both indexes have been raised/lowered, and raising/lowering is done with the metric tensor. Therefore, if $G$ contains the components of $g_{\mu\,\nu}$ with $\mu$ naming the rows of the matrix representation, then the matrix that contains the ${U_{\mu}}^\sigma$ is $G\,U\,G^{-1}$. The rows of this entity correspond to the index $\mu$ in ${U_{\mu}}^\sigma$, and we need to contract the $\sigma$ index (which names the columns) with that in the ${V^\nu}_\sigma\,{U^\gamma}_\nu$ entity. But the $\sigma$ also names the columns in $U\,V$ which is the matrix representation of  ${V^\nu}_\sigma\,{U^\gamma}_\nu$. So we must transpose it before multiplication and thus, finally, the required matrix is:
$$G\,U\,G^{-1}\,(U\,V)^T = G\,U\,G^{-1}\,V^T\,U$$
Again, you should work this out in detail with $2\times2$ examples to understand it fully.
