This is something standard, by I'm trying to redo this with spectral theory. Suppose we start with the usual postulates of quantum mechanics:
States are unit rays on a separable Hilbert space. In particular they are described by unit vectors.
Observables are hermitian operators acting on the Hilbert space.
The possible values of an observable are the ones from its spectrum.
If $A$ is an observable with associated projection-valued measure $\mathbb{P}_A$ given by the spectral theorem, its values on the state $\Psi$ are described by the probability measure $$\mu_A(E)=(\Psi,\mathbb{P}_A(E)\Psi),\quad \forall \text{ measurable $E\subset \sigma(A)$}$$
If the measurement outcome lies in $E\subset \sigma(A)$, then the post-measurement state is the normalized projection $$\Psi'=\frac{1}{\|\mathbb{P_A(E)}\Psi\|}\mathbb{P}_A(E)\Psi$$
Now, if we assume one classical ensemble of quantum states $\{(p_i,\Psi_i)\}$ with probabilities $p_i$ we can show two things upon defining the density operator $\rho = \sum_i p_i \Psi_i(\Psi_i,\cdot)$:
The probability measure, including the uncertainty in the state becomes $$\mu_{A}(E)=\operatorname{Tr}(\mathbb{P}_A(E)\rho)$$ This follows by classical probability, which states that $$\mu_A(E)=\sum_i p_i\mu_A^i(E)=\sum_i p_i (\Psi_i,\mathbb{P}_A(E)\Psi_i)$$ decomposing $\mathbb{P}_A(E)$ in one orthonormal basis the result follows.
The mean values become $$\langle A\rangle = \operatorname{Tr}(A\rho)$$
this follows directly observing that if $f : \sigma(A)\to \mathbb{C}$ is integrable against the measures $\mu_A$ and $\mathbb{P}_A$ then $$\int_{\sigma(A)} f(\lambda)d\mu_A(\lambda)=\operatorname{Tr}\int_{\sigma(A)}f(\lambda)d\mathbb{P}_A(\lambda)\rho$$
and applying to $f(\lambda)=\lambda$ which gives the mean value.
So this line of thought is working fine, because these are the correct expressions. Now I wanted to derive similarly the post-measurement state when the result of the measurement lies in $E\subset \sigma(A)$.
My line of thought was: if the state of the system is $\Psi_i$ the post-measurement state is $\Psi_i'$ given by postulate (5). This will happen with probability $p_i$. So we get a new ensemble of states $\{(p_i,\Psi_i')\}$ with same probabilities.
If we assemble the density operator we have
$$\rho'=\sum_i p_i \Psi_i'(\Psi_i',\cdot)=\sum_i p_i \frac{1}{\|\mathbb{P}_A(E)\Psi_i\|^2}\mathbb{P}_A(E)\Psi_i(\mathbb{P}_A(E)\Psi_i,\cdot).$$
Since the projectors are hermitian we get
$$\rho'=\mathbb{P}_A(E)\sum_i p_i \frac{1}{\|\mathbb{P}_A(E)\Psi_i\|^2}\Psi_i(\Psi_i,\cdot)\mathbb{P}_A(E).$$
But now I cant proceed because of the denominator term. I mean, If I'm not mistaken, the result should be
$$\rho'=\frac{1}{\operatorname{Tr}\mathbb{P}_A(E)\rho}\mathbb{P}_A(E)\rho\mathbb{P}_A(E)$$
What am I missing here? How can we arrive at the correct form for the post-selected density operator following the above reasoning?