Was Dirac's discovery of the Dirac equation by requiring a positive-definite probability density $j^0$ an unjustified coincidence?

Question

Ever since I learned about the Dirac equation as an undergraduate student, it bothered me that it was introduced - both in my course and many others - on the back of concluding that the time component of the conserved current density $j^0$ for the Klein-Gordon equation was not positive definite, and so could not be a probability density in the way that might be hoped for a relativistic upgrade of the Schrödinger equation, only for this issue to be immediately thrown away altogether by then jumping to quantum field theory.

There are plenty of other posts that touch on the existence of this positive-definite Dirac $j^0$, such as here, here and here.

Having read the introductions of Weinberg's books on quantum mechanics and QFT Volume 1, I now realise that this strange feature of my course was a product of history. Dirac originally looked for a relativistic wave equation which had a positive-definite $j^0$. At the time, Dirac concluded that the lack of a positive-definite $j^0$ for the K-G equation really meant that relativistic particles could not be described by it.

Other issues that still plagued this new equation, such as dealing with the negative energy states, eventually led to the invention of QFT. There's also this post which points out that the interpretation of the Dirac spinors as quantum states is inherently problematic.

With all of this in mind, is it therefore just a coincidence that Dirac's original reasoning led to the discovery of an equation describing relativistic particles? In other words, with the benefit of hindsight, was Dirac's original requirement that $j^0$ be positive-definite justified or important?

Although I believe one could construct a reason for why it does mathematically, from the answer under this post and the relationship between the positive-definiteness of $j^0$ and the equation being first-order in time rather than second-order (EDIT: also see the answer below from J.G.), this doesn't seem to actually have any bearing at all on the second quantisation procedure that emerged soon after historically, or indeed on QFT in general.

Answer 1

Since the Dirac equation improves our perspective on $E<0$ solutions (thereby predicting spin-$1/2$ electrons and positrons), a less historical motivation would make it look like antimatter and spin are put in by fiat, as the latter is in pre-relativistic QM. But this can leave a student with your impression:

is it therefore just a coincidence that the Dirac equation has a positive-definite $j^0$ with respect to its discovery as an equation describing relativistic particles?

The free-particle TDSE $i\partial_t\psi=m\psi$ (where we interpret mass as energy to anticipate relativity) should have a Lorentz-invariant generalization of the form $i\gamma^\mu\partial_\mu\psi=m\psi$, from which we can prove $j^\mu:=\psi^\dagger\gamma^0\gamma^\mu\psi$ is a conserved $4$-current, with $j^0=\psi^\dagger\psi\ge0$. You're essentially asking whether this theorem is "coincidental". I'd say no, because we're generalizing a Galilean-invariant equation that was already known to imply a $\rho\ge0$ continuity equation, rather than starting from the relativity-inspired KGE. In other words, we don't have to pass through a $\partial_t^2$ experiment that would doom us to $E^2=\cdots\implies j^0\propto E=\pm\sqrt{\cdots}$ from the start.