In a diatomic molecule, the nuclear potential in the ground and excited states can take different shapes. The coordinates might be displaced and the curvatures might not be the same. For example, we may have
$$ H_g=\frac{p^2_g}{2m_g} + \frac{1}{2}m_g\omega_g^2x_g^2 \\ H_e=\frac{p^2_e}{2m_e} + \frac{1}{2}m_e\omega_e^2x_e^2 $$
The coordinates are usually related by $x_g = x_e - x_0$ where $x_0$ is a constant and since momentum is just derivative of position, it follows that $p_g=p_e$. We can then proceed by second quantization:
$$ x_g=\sqrt{\frac{\hbar}{2m_g\omega_g}}(b_g^{\dagger}+b_g) \\ x_e=\sqrt{\frac{\hbar}{2m_e\omega_e}}(b_e^{\dagger}+b_e) $$
and the number state of ground can be written as $|n_g\rangle$ while for excited state $|n_e\rangle$.
The Franck-Condon factor $\langle n_g|n_e \rangle$ was derived analytically and algebraically in many papers. However, why is one allowed to calculate $\langle n_g|n_e \rangle$ in the first place? Because from my understanding, these are not the same harmonic oscillators since they have different shapes and we second quantized them using different operators. Doesn't this mean that $|n_g\rangle$ and $|n_e\rangle$ live in different Hilbert space and thus $\langle n_g|n_e \rangle=0$ no matter what?
I know I must be wrong here because there are plenty existing works on this but I just don't see how we are allowed to calculate overlap integral between $|n_g\rangle$ and $|n_e\rangle$. I would really appreciate if someone could help clarify.