How to understand the trace operation below?

Question

When studying the density operator, I read the document below. But the following trace calculation confuse me. It seems what he do is $\mathrm{tr}(\langle\psi _{a}|A|\psi _{a}\rangle)=\mathrm{tr}(\langle\psi_{a}| )\mathrm{tr}(A)\mathrm{tr}(|\psi_{a}\rangle)=\mathrm{tr}(A|\psi _{a}\rangle\langle\psi _{a}|)$ since he writes "cyclic property of the trace". But I think trace is only defined for square matrix, how can we use things like $\langle\psi _{a}|$?

Answer 1

I think that by $''$cyclic property of the trace$''$ we mean the following :

Consider $\:k\:$ finite dimensional matrices \begin{align} \mathrm A_1 & \boldsymbol{=} n_1 \times n_2 \quad \texttt{matrix} \tag{1-01}\label{1-01}\\ \mathrm A_2 & \boldsymbol{=} n_2 \times n_3 \quad \texttt{matrix} \tag{1-02}\label{1-02}\\ \mathrm A_3 & \boldsymbol{=} n_3 \times n_4 \quad \texttt{matrix} \tag{1-03}\label{1-03}\\ \cdots & \boldsymbol{=} \cdot\cdot\: \times \cdot\cdot\: \quad \texttt{matrix} \tag{1-....}\label{1-....}\\ \mathrm A_k & \boldsymbol{=} n_k \times n_1 \quad \texttt{matrix} \tag{1-0k}\label{1-0k} \end{align} Then the product matrix $\:\mathrm A_1\mathrm A_2\mathrm A_3\cdots\mathrm A_k\:$ and all cyclic permutations of it are square matrices. Explicitly \begin{align} \mathrm A_1\mathrm A_2\mathrm A_3\cdots\mathrm A_k & \boldsymbol{=} n_1 \times n_1 \quad \texttt{square matrix} \tag{2-01}\label{02-01}\\ \mathrm A_2\mathrm A_3\mathrm A_4\cdots\mathrm A_1 & \boldsymbol{=} n_2 \times n_2 \quad \texttt{square matrix} \tag{2-02}\label{02-02}\\ \mathrm A_3\mathrm A_4\mathrm A_5\cdots\mathrm A_2 & \boldsymbol{=} n_3 \times n_3 \quad \texttt{square matrix} \tag{2-03}\label{01-03}\\ \cdots & \boldsymbol{=} \cdot\cdot\: \times \cdot\cdot\: \quad \texttt{square matrix} \tag{2-....}\label{01-....}\\ \mathrm A_{\rho}\mathrm A_{\rho\boldsymbol{+}1}\mathrm A_{\rho\boldsymbol{+}2}\cdots\mathrm A_{\rho\boldsymbol{-}1} & \boldsymbol{=} n_\rho \times n_\rho \quad \texttt{square matrix} \tag{2-0k}\label{01-0k} \end{align}

Then \begin{equation} \texttt{Tr}\left(\mathrm A_1\mathrm A_2\mathrm A_3\cdots\mathrm A_k\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm A_2\mathrm A_3\mathrm A_4\cdots\mathrm A_1\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm A_3\mathrm A_4\mathrm A_5\cdots\mathrm A_2\right)\boldsymbol{=}\cdots \tag{3}\label{3} \end{equation} Under this spirit if \begin{align} \mathrm A_1 & \boldsymbol{\equiv} \langle\psi|\boldsymbol{=} 1 \times n \boldsymbol{=} \texttt{one row matrix} \tag{4-01}\label{04-01}\\ \mathrm A_2 & \boldsymbol{\equiv} A \boldsymbol{=} n \times n \quad \texttt{square matrix} \tag{4-02}\label{04-02}\\ \mathrm A_3 & \boldsymbol{\equiv} |\psi\rangle\boldsymbol{=} n \times 1 \boldsymbol{=} \texttt{one column matrix} \tag{4-03}\label{04-03} \end{align} then
\begin{equation} \texttt{Tr}\left(\mathrm A_1\mathrm A_2\mathrm A_3\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm A_2\mathrm A_3\mathrm A_1\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm A_3\mathrm A_1\mathrm A_2\right) \tag{5}\label{5} \end{equation} is translated to \begin{equation} \texttt{Tr}\left(\mathrm \langle\psi|A\mathrm |\psi\rangle\right)\boldsymbol{=}\texttt{Tr}\left(A\mathrm |\psi\rangle\mathrm \langle\psi|\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm |\psi\rangle\mathrm \langle\psi|A\right) \tag{6}\label{6} \end{equation} But in any case is not permissible to write \begin{equation} \texttt{Tr}\left(\mathrm A_1\mathrm A_2\mathrm A_3\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm A_1\right)\texttt{Tr}\left(\mathrm A_2\right)\texttt{Tr}\left(\mathrm A_3\right) \tag{7}\label{7} \qquad \textbf{(wrong !!!)} \end{equation} as OP does in the question since on one hand the trace of non-square matrices has no sense and on the other hand this is wrong even in the case that all matrices are square.

Now, I think that a proof of the validity of the cyclic property \eqref{3} for finite dimensional matrices $\:\rm A_\rho, \rho=1,2,\cdots k\:$ will help us to sketch a proof as an answer to the question. It's easy to realize that it's sufficient to prove \eqref{3} for two matrices, say $\:\rm P=\{\rm p_{ij}\}\:$ and $\:\rm Q=\{q_{k\ell}\}\:$ of dimensions $\:n\times m\:$ and $\:m\times n\:$ respectively \begin{equation} \texttt{Tr}\left(\mathrm P\mathrm Q\right)\boldsymbol{=}\texttt{Tr}\left(\mathrm Q\mathrm P\right) \tag{8}\label{8} \end{equation} since for any number $\:k\:$ of matrices \begin{equation} \texttt{Tr}\left(\underbrace{\mathrm A_1}_{\rm P}\underbrace{\mathrm A_2\mathrm A_3\cdots\mathrm A_k}_{\rm Q}\right)\boldsymbol{=}\texttt{Tr}\left(\underbrace{\mathrm A_2\mathrm A_3\cdots\mathrm A_k}_{\rm Q}\underbrace{\mathrm A_1}_{\rm P}\right) \tag{9}\label{9} \end{equation} The proof of \eqref{8} runs as follows \begin{equation} \texttt{Tr}\left(\mathrm P\mathrm Q\right)\boldsymbol{=}\sum\limits_{i=1}^{i=n}\sum\limits_{j=1}^{j=m}\mathrm p_{ij}\mathrm q_{ji}\boldsymbol{=}\sum\limits_{j=1}^{j=m}\sum\limits_{i=1}^{i=n}\mathrm q_{ji}\mathrm p_{ij}\boldsymbol{=}\texttt{Tr}\left(\mathrm Q\mathrm P\right) \tag{10}\label{10} \end{equation}

We sketch now a quick proof of equation \eqref{6} in the special case of the 3-dimensional Hilbert space $\:\mathbb C^{3}$. Later on this proof will be generalized for an infinite-dimensional separable Hilbert space (with infinite countable complete orthonormal basis $\:\{\phi_\mu\}$).

For the state vector $\:|\psi\rangle\:$ we have \begin{align} |\psi\rangle & \boldsymbol{=} z_1\,\phi_1\boldsymbol{+}z_2\,\phi_2\boldsymbol{+}z_3\,\phi_3\boldsymbol{=}\sum\limits_{\mu=1}^{\mu=3}z_\mu\,\phi_\mu\,,\quad z_\mu\boldsymbol{=}\langle\phi_\mu|\psi\rangle \in \mathbb C \tag{11a}\label{11a}\\ & \texttt{so} \quad\quad |\psi\rangle \boldsymbol{=} \sum\limits_{\mu=1}^{\mu=3}|\phi_\mu\rangle\langle\phi_\mu|\psi\rangle \tag{11b}\label{11b}\\ \langle\psi| & \boldsymbol{=} \overline{z_1}\,\phi_1\boldsymbol{+}\overline{z_2}\,\phi_2\boldsymbol{+}\overline{z_3}\,\phi_3\boldsymbol{=}\sum\limits_{\mu=1}^{\mu=3}\overline{z_\mu}\,\phi_\mu\,,\quad \overline{z_\mu}\boldsymbol{=}\overline{\langle\phi_\mu|\psi\rangle}\boldsymbol{=}\langle\psi|\phi_\mu\rangle \in \mathbb C \tag{11c}\label{11c}\\ & \texttt{so} \quad\quad \langle\psi|\boldsymbol{=} \sum\limits_{\mu=1}^{\mu=3}\langle\phi_\mu|\langle\psi|\phi_\mu\rangle\boldsymbol{=}\sum\limits_{\mu=1}^{\mu=3}\langle\phi_\mu|\overline{\langle\phi_\mu|\psi\rangle} \tag{11d}\label{11d} \end{align} Formally we could consider them as one-column and one-row matrices respectively \begin{equation} |\psi\rangle \boldsymbol{=} \begin{bmatrix} \:z_1 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_2 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_3 \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix}\,,\qquad \langle\psi| \boldsymbol{=} \begin{bmatrix} \:\overline{z_1} \: & \:\overline{z_2} \: & \:\overline{z_3} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \tag{12}\label{12} \end{equation} Consider that the operator $\:A\:$ is represented by the following $\:3\times 3\:$ complex matrix \begin{equation} A\boldsymbol{=} \begin{bmatrix} a_{11} & a_{12} & a_{13} \vphantom{\dfrac{a}{b}}\:\\ a_{21} & a_{22} & a_{23} \vphantom{\dfrac{a}{b}}\:\\ a_{31} & a_{32} & a_{33} \vphantom{\dfrac{a}{b}} \end{bmatrix}\,,\quad a_{ij} \in \mathbb C \tag{13}\label{13} \end{equation} Now we have \begin{equation} \rm P_{\psi}\boldsymbol{=}|\psi\rangle\langle\psi|\boldsymbol{=} \begin{bmatrix} \:z_1 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_2 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_3 \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \begin{bmatrix} \:\overline{z_1} \: & \:\overline{z_2} \: & \:\overline{z_3} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix}\boldsymbol{=} \begin{bmatrix} z_1\overline{z_1} & z_1\overline{z_2} & z_1\overline{z_3} \vphantom{\dfrac{a}{b}}\:\\ z_2\overline{z_1} & z_2\overline{z_2} & z_2\overline{z_3} \vphantom{\dfrac{a}{b}}\:\\ z_3\overline{z_1} & z_3\overline{z_2} & z_3\overline{z_3} \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \tag{14}\label{14} \end{equation} We use the symbol $\:\rm P_{\psi}\:$ for the operator $\:|\psi\rangle\langle\psi|\:$ because if $\:|\psi\rangle\:$ is a normalized state vector,$\:\langle\psi|\psi\rangle\boldsymbol{=}1$, then $\:\rm P_{\psi}\:$ is the $''$projector$''$ on the direction $\:|\psi\rangle\:$. The operator $\:A\,|\psi\rangle\langle\psi|\:$ is represented by the matrix \begin{equation} A\,|\psi\rangle\langle\psi|\boldsymbol{=} \begin{bmatrix} a_{11} & a_{12} & a_{13} \vphantom{\dfrac{a}{b}}\:\\ a_{21} & a_{22} & a_{23} \vphantom{\dfrac{a}{b}}\:\\ a_{31} & a_{32} & a_{33} \vphantom{\dfrac{a}{b}} \end{bmatrix}\, \begin{bmatrix} z_1\overline{z_1} & z_1\overline{z_2} & z_1\overline{z_3} \vphantom{\dfrac{a}{b}}\:\\ z_2\overline{z_1} & z_2\overline{z_2} & z_2\overline{z_3} \vphantom{\dfrac{a}{b}}\:\\ z_3\overline{z_1} & z_3\overline{z_2} & z_3\overline{z_3} \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \tag{15}\label{15} \end{equation} that is ^{\begin{equation} A\,|\psi\rangle\langle\psi|\boldsymbol{=} \begin{bmatrix} \begin{array}{c|c|c} a_{11}z_1\overline{z_1}\boldsymbol{+}a_{12}z_2\overline{z_1}\boldsymbol{+}a_{13}z_3\overline{z_1} & \cdots & \cdots \vphantom{\dfrac{\tfrac{a}{b}}{\tfrac{a}{b}}}\\ \hline \cdots & a_{21}z_1\overline{z_2}\boldsymbol{+}a_{22}z_2\overline{z_2}\boldsymbol{+}a_{23}z_3\overline{z_2} & \cdots \vphantom{\dfrac{\tfrac{a}{b}}{\tfrac{a}{b}}}\\ \hline \cdots & \cdots & a_{31}z_1\overline{z_3}\boldsymbol{+}a_{32}z_2\overline{z_3}\boldsymbol{+}a_{33}z_3\overline{z_3} \vphantom{\dfrac{\tfrac{a}{b}}{\tfrac{a}{b}}} \end{array} \end{bmatrix} \tag{16}\label{16} \end{equation}} We show only the diagonal elements since we are interested for their sum \begin{align} \texttt{Tr}\left(A|\psi\rangle\langle\psi|\right) \boldsymbol{=} & \hphantom{ ...}\left(a_{11}z_1\overline{z_1}\boldsymbol{+}a_{12}z_2\overline{z_1}\boldsymbol{+}a_{13}z_3\overline{z_1}\right) \nonumber\\ & \boldsymbol{+}\left(a_{21}z_1\overline{z_2}\boldsymbol{+}a_{22}z_2\overline{z_2}\boldsymbol{+}a_{23}z_3\overline{z_2}\right) \nonumber\\ & \boldsymbol{+}\left(a_{31}z_1\overline{z_3}\boldsymbol{+}a_{32}z_2\overline{z_3}\boldsymbol{+}a_{33}z_3\overline{z_3}\right) \nonumber\\ \boldsymbol{=} & \hphantom{ ...\,}\overline{z_1}\left(a_{11}z_1\boldsymbol{+}a_{12}z_2\boldsymbol{+}a_{13}z_3\right) \nonumber\\ & \boldsymbol{+}\overline{z_2}\left(a_{21}z_1\boldsymbol{+}a_{22}z_2\boldsymbol{+}a_{23}z_3\right) \nonumber\\ & \boldsymbol{+}\overline{z_3}\left(a_{31}z_1\boldsymbol{+}a_{32}z_2\boldsymbol{+}a_{33}z_3\right) \nonumber\\ \boldsymbol{=} & \begin{bmatrix} \:\overline{z_1} \: & \:\overline{z_2} \: & \:\overline{z_3} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \begin{bmatrix} \:a_{11}z_1\boldsymbol{+}a_{12}z_2\boldsymbol{+}a_{13}z_3 \: \vphantom{\dfrac{a}{b}}\:\\ \:a_{21}z_1\boldsymbol{+}a_{22}z_2\boldsymbol{+}a_{23}z_3 \: \vphantom{\dfrac{a}{b}}\:\\ \:a_{31}z_1\boldsymbol{+}a_{32}z_2\boldsymbol{+}a_{33}z_3 \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \nonumber\\ \boldsymbol{=} & \begin{bmatrix} \:\overline{z_1} \: & \:\overline{z_2} \: & \:\overline{z_3} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \begin{bmatrix} a_{11} & a_{12} & a_{13} \vphantom{\dfrac{a}{b}}\:\\ a_{21} & a_{22} & a_{23} \vphantom{\dfrac{a}{b}}\:\\ a_{31} & a_{32} & a_{33} \vphantom{\dfrac{a}{b}} \end{bmatrix} \begin{bmatrix} \:z_1 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_2 \: \vphantom{\dfrac{a}{b}}\:\\ \:z_3 \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix} \nonumber\\ \boldsymbol{=} & \:\langle\psi|A|\psi\rangle \tag{17}\label{17} \end{align} qed.

$=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!$

Finally we run the proof for an infinite-dimensional separable Hilbert space $\:\mathbb H\:$ with infinite countable complete orthonormal basis $\:\{\phi_\mu\}\:$ having in mind (or under the table) the finite case above.

First we must say that if $\:\mathcal O\:$ is an operator on the Hilbert space $\:\mathbb H\:$ then its trace is defined by \begin{equation} \texttt{Tr}\left(\mathcal O \right) \boldsymbol{\equiv} \langle\phi_\mu|\mathcal O|\phi_\mu\rangle \, \quad \texttt{(Einstein summation convention)} \tag{18}\label{18} \end{equation} The trace is well-defined : it actually depends on $\:\mathcal O\:$ and not on the basis $\:\{\phi_\mu\}\:$ used. In other words it's invariant.

The states $\:|\psi\rangle\:$ and $\:\langle\psi|\:$ are expressed in the $\:\{\phi_\mu\}\:$ basis as follows
\begin{align} |\psi\rangle & \boldsymbol{=} |\phi_\mu\rangle\:\langle\phi_\mu|\psi\rangle \tag{19a}\label{19a}\\ \langle\psi| & \boldsymbol{=} \langle\phi_\nu|\:\overline{\langle\phi_\nu|\psi\rangle} \tag{19b}\label{19b} \end{align} corresponding to equations \eqref{11b},\eqref{11d} of the finite 3-dimensional case, while the complex numbers $\:\langle\phi_\mu|\psi\rangle,\overline{\langle\phi_\nu|\psi\rangle}\:$ correspond to the coordinates $\:z_\mu,\overline{z_\nu}$.

So \begin{equation} \rm P_{\psi}\boldsymbol{=}|\psi\rangle\langle\psi|\boldsymbol{=} \Bigl(|\phi_\mu\rangle\langle\phi_\mu|\psi\rangle\Bigr) \Bigl(\langle\phi_\nu|\overline{\langle\phi_\nu|\psi\rangle}\Bigr)\quad\boldsymbol{\implies} \nonumber \end{equation} \begin{equation} \rm P_{\psi}\boldsymbol{=}|\psi\rangle\langle\psi|\boldsymbol{=} \Bigl(\langle\phi_\mu\psi\rangle\: \overline{\langle\phi_\nu|\psi\rangle}\vphantom{\dfrac{a}{b}}\Bigr) \:|\phi_\mu\rangle\langle\phi_\nu|\boldsymbol{=} |\phi_\mu\rangle\Bigl(\langle\phi_\mu\psi\rangle\: \overline{\langle\phi_\nu|\psi\rangle}\vphantom{\dfrac{a}{b}}\Bigr) \:\langle\phi_\nu| \tag{20}\label{20} \end{equation} Formally the operator $\:\rm P_{\psi}\:$ could be represented by a $''$square$''$ matrix of infinite but countable rows and columns as follows \begin{equation} \rm P_{\psi}\boldsymbol{=}|\psi\rangle\langle\psi|\boldsymbol{=} \begin{bmatrix} \begin{array}{c|c|c|c|c} \langle\phi_1|\psi\rangle |\overline{\langle\phi_1|\psi\rangle} & \langle\phi_1|\psi\rangle |\overline{\langle\phi_2|\psi\rangle} & \boldsymbol{\cdots} & \langle\phi_1|\psi\rangle |\overline{\langle\phi_\nu|\psi\rangle} & \boldsymbol{\cdots}\vphantom{\dfrac{a}{b}}\:\\ \langle\phi_2|\psi\rangle |\overline{\langle\phi_1|\psi\rangle} & \langle\phi_2|\psi\rangle |\overline{\langle\phi_2|\psi\rangle} & \boldsymbol{\cdots} & \langle\phi_2|\psi\rangle |\overline{\langle\phi_\nu|\psi\rangle} & \boldsymbol{\cdots}\vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} \vphantom{\dfrac{a}{b}}\:\\ \langle\phi_\mu|\psi\rangle |\overline{\langle\phi_1|\psi\rangle} & \langle\phi_\mu|\psi\rangle |\overline{\langle\phi_2|\psi\rangle} & \boldsymbol{\cdots} & \langle\phi_\mu|\psi\rangle |\overline{\langle\phi_\nu|\psi\rangle} & \boldsymbol{\cdots}\vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} & \boldsymbol{\cdots} \vphantom{\dfrac{a}{b}}\:\\ \end{array} \end{bmatrix} \tag{21}\label{21} \end{equation} corresponding to the matrix in the last to the right side of equation \eqref{14} for the finite case.

From the definition of the trace, see equation \eqref{18} \begin{align} \texttt{Tr}\left(A|\psi\rangle\langle\psi|\right) \boldsymbol{=} & \Big<\phi_\nu\vphantom{\dfrac{a}{b}}\Big| A|\psi\rangle\langle\psi|\Big|\phi_\nu\Big>\boldsymbol{=}\Big<\phi_\nu\vphantom{\dfrac{a}{b}}\Big| A\,\Big|\psi\langle\psi|\phi_\nu\rangle\Big>\boldsymbol{=}\Big<\phi_\nu\vphantom{\dfrac{a}{b}}\Big| A\,\Big| \overline{\langle\phi_\nu|\psi\rangle}\psi\Big> \nonumber\\ \boldsymbol{=} & \Big<\phi_\nu\overline{\langle\phi_\nu|\psi\rangle}\vphantom{\dfrac{a}{b}}\Big| A\,\Big| \psi\Big>\boldsymbol{=}\langle\psi|A|\psi\rangle \tag{22}\label{22} \end{align} QED.

$=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!=\!$

\begin{align} & \overbrace{\begin{bmatrix} \boldsymbol{\longleftarrow} \langle\phi_\nu| \boldsymbol{\longrightarrow} \vphantom{\tfrac{a}{b}} \end{bmatrix}\boldsymbol{\cdot} \begin{bmatrix} \boldsymbol{\nwarrow} & \boldsymbol{\uparrow} & \boldsymbol{\nearrow} \vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\leftarrow} & A & \boldsymbol{\rightarrow} \vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\swarrow} & \boldsymbol{\downarrow} & \boldsymbol{\searrow} \vphantom{\dfrac{a}{b}} \end{bmatrix}\boldsymbol{\cdot} \begin{bmatrix} \:\boldsymbol{\uparrow} \: \vphantom{\dfrac{a}{b}}\:\\ \:|\psi\rangle \: \vphantom{\dfrac{a}{b}}\:\\ \:\boldsymbol{\downarrow} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix}\boldsymbol{\cdot} \underbrace{ \begin{bmatrix} \boldsymbol{\longleftarrow} \langle\psi|\boldsymbol{\longrightarrow} \vphantom{\tfrac{a}{b}} \end{bmatrix}\boldsymbol{\cdot} \begin{bmatrix} \:\boldsymbol{\uparrow} \: \vphantom{\dfrac{a}{b}}\:\\ \:|\phi_{\nu}\rangle \: \vphantom{\dfrac{a}{b}}\:\\ \:\boldsymbol{\downarrow} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix}}_{\texttt{scalar}\boldsymbol{=}\langle\psi|\phi_\nu\rangle \boldsymbol{=}\overline{\langle\phi_\nu|\psi\rangle}}}^{\Big<\phi_\nu\vphantom{\dfrac{a}{b}}\Big| A|\psi\rangle\langle\psi|\Big|\phi_\nu\Big>} \nonumber\\ & \tag{23}\label{23}\\ &\boldsymbol{=} \underbrace{\underbrace{\begin{bmatrix} \boldsymbol{\longleftarrow} \langle\phi_\nu|\,\overline{\langle\phi_\nu|\psi\rangle}\boldsymbol{\longrightarrow} \vphantom{\tfrac{a}{b}} \end{bmatrix}}_{\langle\psi|} \boldsymbol{\cdot} \begin{bmatrix} \boldsymbol{\nwarrow} & \boldsymbol{\uparrow} & \boldsymbol{\nearrow} \vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\leftarrow} & A & \boldsymbol{\rightarrow} \vphantom{\dfrac{a}{b}}\:\\ \boldsymbol{\swarrow} & \boldsymbol{\downarrow} & \boldsymbol{\searrow} \vphantom{\dfrac{a}{b}} \end{bmatrix}\boldsymbol{\cdot} \begin{bmatrix} \:\boldsymbol{\uparrow} \: \vphantom{\dfrac{a}{b}}\:\\ \:|\psi\rangle \: \vphantom{\dfrac{a}{b}}\:\\ \:\boldsymbol{\downarrow} \: \vphantom{\dfrac{a}{b}}\: \end{bmatrix}}_{\langle\psi|A|\psi\rangle} \nonumber \end{align}

Answer 2

Note that $x=\langle\psi|\hat{A}|\psi\rangle$ is just a number. (I dropped the subscript $a$ as it does not seem to play any role.) Therefore, tr$\{x\}=x$. The trace of an operator $\hat{O}$ is formally defined in terms of any complete orthogonal basis $|n\rangle$ as $$ \text{tr}\{\hat{O}\} = \sum_n \langle n|\hat{O}|n\rangle . $$ Any complete basis can be used to resolve the identity $$ \mathbf{1} = \sum_n |n\rangle\langle n| . $$ Now we can insert the resolved identity into the expression for $x$: $$ x=\sum_n\langle\psi|n\rangle\langle n|\hat{A}|\psi\rangle = \sum_n\langle n|\hat{A}|\psi\rangle\langle\psi|n\rangle = \text{tr}\{\hat{A}|\psi\rangle\langle\psi|\} . $$ All we did was to exchange the two numbers under the summation to obtain the formal expression for the trace.