Meaning of expectation value of product of non-commuting operators

Let $\hat{A}$ and $\hat{B}$ be Hermitian observables with spectra labeled by $a$ and $b$. Then we can write $$\hat{A} = \sum_a a\, \hat{P}_a$$ $$\hat{B} = \sum_b b\, \hat{P}_b$$

where $\hat{P}_a$ and $\hat{P}_b$ are the eigenspace projection operators for $\hat{A}$ and $\hat{B}$.

Let $|\psi \rangle$ be a state and consider the expectation value \begin{align} \langle\psi|\hat{A} \hat{B}|\psi \rangle = \sum_{a,b} ab\,\langle \psi|\hat{P}_a \hat{P}_b |\psi\rangle. \end{align}

If $[\hat{A},\hat{B}] = 0$, then it is clear how to interpret $\langle\psi|\hat{A} \hat{B}|\psi \rangle$: when we perform a simultaneous measurement of $\hat{A}$ and $\hat{B}$, we project the state onto a simultaneous eigenspace and measure the corresponding eigenvalues $a,b$. The probability that the state will be projected onto the $a,b$ eigenspace is simply $\langle \psi|\hat{P}_a \hat{P}_b |\psi\rangle$. Then from the above equation, we see that $\langle\psi|\hat{A} \hat{B}|\psi \rangle$ represents the average value of the product of the measurements, taken over a large number of repeated experiments.

My question concerns the meaning of $\langle\psi|\hat{A} \hat{B}|\psi \rangle$ when $\hat{A}$ and $\hat{B}$ do not commute. My first thought is that it represents the average value of the product of the measurements when $\hat{B}$ is measured first, followed by a measurement of $\hat{A}$. But this quantity, it seems to me, should instead be computed as follows:

1. First make a measurement of $\hat{B}$. This projects the state onto an eigenspace with eigenvalue $b$ with probability $\langle \psi|\hat{P}_b |\psi \rangle$. After making this measurement, our new state is now $$|\psi'\rangle = \frac{1}{\sqrt{\langle\psi |\hat{P_b} |\psi \rangle}}\hat{P}_b|\psi\rangle$$

2. We now perform a measurement of $\hat{A}$ on $|\psi'\rangle$. This projects onto an eigenspace with eigenvalue $a$ with probability $\langle \psi'|\hat{P}_a|\psi'\rangle$

3. Hence, the probability of obtaining first an eigenvalue $b$ followed by an eigenvalue $a$ is

$$\langle \psi'|\hat{P}_a|\psi'\rangle\,\langle \psi|\hat{P}_b|\psi\rangle = \langle \psi|\hat{P}_b \hat{P}_a \hat{P}_b|\psi\rangle$$

1. And thus I would expect the average value of the product of the measurements to be $$\sum_{a,b} ab\,\langle \psi|\hat{P}_b \hat{P}_a \hat{P}_b|\psi\rangle.$$

If this reasoning is correct, then how am I to interpret $\langle \psi | \hat{A} \hat{B} |\psi \rangle$ (since it is clearly not equal to the result in bullet point 4 when the projectors do not commute)? If the reasoning is incorrect, what is wrong with it?

Right, in general you're not going to see a straightforward equivalence there. We can use Dirac notation with $\hat P_b = |b\rangle\langle b|$ to see that $\langle \hat A \hat B \rangle = \sum_{a,b} a~b~\langle \psi | a \rangle~\langle a | b \rangle~\langle b | \psi \rangle$ and even inserting an identity matrix for $b$ (call it $b'$) gives: \begin{align}\langle \hat A \hat B \rangle =& \sum_{a,b,b'} a~b~\langle \psi | b'\rangle~\langle b'|a \rangle~\langle a | b \rangle~\langle b | \psi \rangle\\=&\sum_{b,b'} b~\psi^*(b')~\psi(b) ~ \langle b'|\hat A| b\rangle\end{align}Indeed, you need to insert some sort of $\delta_{bb'}$ into this last sum to get the $b ~ \langle b|\hat A | b\rangle$ sense of "measure $B$ first, then $A$," which this expression doesn't have unless it's hidden in that $\hat A$ term.
There is a very simple reason why you do not see this straightforward equivalence. Let's work in a finite-dimensional Hilbert space $\psi \in \mathbb C^N.$ Then the matrix $\hat C = \hat A \hat B$ is really given by the Einstein sum $$C_{ik} = A_{ij} ~B_{jk}.$$This is Hermitian if and only if $C_{ik}^* = C_{ki}$ but the complex conjugate here is$$C_{ik}^* = A_{ij}^* ~B_{jk}^* = B_{kj}~A_{ji}$$and demanding that this is equal to $C_{ki}$ is therefore demanding that $B_{kj} A_{ji} = A_{kj} B_{ji}$ or therefore $[\hat A, \hat B] = 0.$
In other words, the product of two Hermitian matrices is only Hermitian if they commute. In general the expectation $\langle \hat A \hat B \rangle$ is going to be a complex number when they do not commute.
If you want something which is Hermitian (say you have a classical expression involving $\langle x~ p\rangle$ that you want to generalize into the quantum case) then you will probably do a symmetric product $\frac 12 (\hat A \hat B + \hat B \hat A)$, which is then again Hermitian if its constituent matrices are.