The explanations of quantizing the Electric/Magnetic (E/M) fields that I've read have all basically worked by using the Coulomb gauge in free space to define the vector potential in some volume as
$$ \mathbf{A}(\mathbf{x},t)=\sum_{\mathbf{k},r}\sqrt{\frac{\hbar c^2}{2V\omega_{\mathbf{k}}}}\mathbf{e}_r(\mathbf{k})[a_{r,\mathbf{k}}e^{i(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t)}+a^*_{r,\mathbf{k}}e^{-i(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t)}] $$
where the classical amplitudes $a$, $a^*$ are essentially engineered to be directly replaced with the creation/annihilation operators $\hat{a}$, $\hat{a}^\dagger$ to give the quantized form of the fields, the Hamiltonian etc. This is neat and easy to follow but I don't see any physical motivation for replacing those particular objects with the creation/annihilation operators.
By contrast, in most pedagogical treatments, the quantum harmonic oscillator is first given in terms of the physical quantities position and momentum, these are then replaced with their quantum versions and the creation/annihilation operators are derived from their anticommutation relations. I find this to be a more satisfying procedure because the $\hat{a}$, $\hat{a}^\dagger$ formalism is shown to naturally emerge from switching to the quantum version of these physical observables. In particular, you have the relations
$$ \hat x = \sqrt{\frac{\hbar}{2m\omega}}(\hat a+\hat a^\dagger),\quad \hat p = i\sqrt{\frac{\hbar m\omega}{2}}(\hat a+\hat a^\dagger), $$
and
$$ \hat a = \sqrt{\frac{m\omega}{2\hbar}}\left(\hat x + \frac{i}{m\omega}\hat p\right), \quad \hat a ^\dagger = \sqrt{\frac{m\omega}{2\hbar}}\left(\hat x - \frac{i}{m\omega}\hat p\right). $$
Naturally I was wondering if there are analogous physical quantities for the EM field where similar relations to the above hold. Hoping to gain some insight, I considered a single mode $\mathbf{k}$ and wrote $$ \hat a = \hat Q + i\hat P $$ which gets us $$ \hat a e^{i(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t)}+\hat a ^\dagger e^{-i(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t)} = 2\hat Q \cos(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t) - 2\hat P \sin(\mathbf{k}\cdot\mathbf{x}-\omega_{\mathbf{k}}t). $$ I now have an expression in terms of hermitian operators but I'm not sure what they are supposed to be nor do I know why we should expect them to have commutation relations similar to position and momentum. Can anyone clear this up for me or point me to a source which treats the quantized EM field in this way?