Orthogonality between the BRST-Closed Subspace and the non-BRST-Closed One

Consider the BRST quantisation of free Maxwell theory, in one of the averaged Lorenz gauges, $$S = \int - \frac{1}{4} F^2 + \frac{1}{2} (\partial\cdot A)^2 + i \bar{C} \partial^2 C.$$

Calling the modes of the ghost $C$ $c$ and the modes of the potential $A_\mu$ $a_\mu$, the state $$p^\mu a_\mu^\dagger |0\rangle = Q_B\ c^\dagger |0\rangle$$ is BRST-exact, and the state $a_0^\dagger |0\rangle$ isn't, as can be seen by the equation $$Q_B a_0^\dagger |0\rangle = p_0 c^\dagger |0\rangle.$$

What I'm confused about is that these two states aren't orthogonal, $$\langle 0| a_0 (p \cdot a^\dagger) |0\rangle = -p^0.$$

Why is this okay?

The reason I'm finding this odd is that I expect a physical Hilbert space to be orthogonal to the extra stuff, since that's the usual understanding of clean separation in vector spaces.

Why is the BRST-closed Hilbert space not cleanly separated in this sense, and in what sense is it cleanly separated (apart from being invariant under time-evolution, of course)?

So I figured it out.

There are two points the question gets wrong:

1. There is no unique choice of a non-BRST-closed subspace.
2. The correctly chosen non-BRST-closed subspace doesn't need to be orthogonal to the entire BRST-closed subspace, only to the particular representative of the BRST cohomology subspace.

For the first point, note that if $|\psi\rangle$ is not BRST closed, neither is $|\psi\rangle + |\chi\rangle$ where $Q_B |\chi\rangle = 0$. In other words, you can add an arbitrary BRST-closed state to a non-closed one and the resultant is still not closed. Because of this, there's no canonical choice of a non-closed subspace.

The nicest way to convince you of the second point is to write down the Hamiltonian for the Lagrangian in the question in terms of oscillators. Calling the two transverse oscilaltors at any given momentum $\vec{k}$ $a_\alpha$, the oscillator corresponding to $A_0$ $a_0$ and the oscillator corresponding to the spatial longitudinal potential $\vec{\nabla} \cdot \vec{A}$ $a_L$, the Hamiltonian is $$H = \int d^3 k \left[ \left\{ \sum_\alpha a_\alpha^\dagger a_\alpha \right\} + \left\{ \bar{c}^\dagger c + c^\dagger \bar{c} \right\} + \frac{1}{2} \left\{ (a_0 + a_L)^\dagger (a_0 - a_L) +(a_0 - a_L)^\dagger (a_0 + a_L) \right\} \right].$$ Now, $(a_0+a_L)$ corresponds to the space-time longitudinal mode which is BRST-exact in this gauge (it is the BRST variation of $\bar{c}$) and $(a_0-a_L)$ is the "fourth polarisation," which is not BRST-closed.

The point to take away from that Hamiltonian is, time evolution mixes the non-BRST-closed states with the BRST-exact states!

But, the non-closed states are orthogonal to the cohomology representatives, which are the transverse oscillators, and time-evolution doesn't mix them, so the physics is fine.

As a parting comment, it seems to me (but I haven't checked properly) that the way to figure out a* correct representative for the non-closed states is to find the analog of the Hamiltonian above and just observe what the oscillator multiplying the exact oscillator is. Doing the correct quantisation can be subtle, but the point is that the exact non-closed mode that decouples into the Kugo-Ojima quartet (the analog of the set $\{c,\bar{c},a_0+a_L,a_0-a_L\}$ in whatever gauge) is the one that needs to be orthogonal to the physical states.