There is no need to take the non-relativist limit; the magnetic moment is a relativistic concept. But to keep things simple, we shall take it anyway.
Consider Dirac's equation in the presence of an (external) electromagnetic field
$$
(i\not\partial+e\not A-m)\psi=0\tag{1}
$$
as obtained by the minimal coupling prescription (aka, covariant derivative).
If we apply $(i\not\partial+e\not A+m)$ to $(1)$ on the left, and expand the product of terms, we get
$$
\begin{aligned}
0&=(i\not\partial+e\not A+m)(i\not\partial+e\not A-m)\psi=\\
&=(-\partial^2-m^2+ie(\not\partial\not A+\not A\not\partial)+e^2A^2)\psi
\end{aligned}\tag{2}
$$
If we take $A^0=0$ (no electric field) and $\dot A^i=0$ (static magnetic field), then it is a matter of (easy) algebra to show that
$$
\not\partial\not A=\nabla\cdot\boldsymbol A-2i\boldsymbol S\cdot\boldsymbol B\tag{3}
$$
where
$$
\boldsymbol S=\frac12\begin{pmatrix}\boldsymbol\sigma&0\\0&\boldsymbol\sigma\end{pmatrix}\tag{4}
$$
Therefore, $(2)$ reads
$$
(\partial^2+m^2-2e\boldsymbol S\cdot\boldsymbol B+\cdots)\psi=0\tag{5}
$$
where $\cdots$ are terms that are quadratic in $\boldsymbol A$ (diamagentic term) and linear in $\boldsymbol A\cdot\nabla$ (orbital dipole moment).
For this, we see that the energy satisfies (why?)
$$
E^2=\boldsymbol p^2+m^2-2e\boldsymbol S\cdot\boldsymbol B+\cdots\tag{6}
$$
Finally, using $\sqrt{1+x}=1+\frac12x+\mathcal O(x^2)$, we see that
$$
E=m+\frac{\boldsymbol p^2}{2m}-\frac{e}{m}\boldsymbol S\cdot\boldsymbol B+\cdots\tag{7}
$$
and therefore, the dipole moment is
$$
\boldsymbol\mu\equiv-\lim_{\boldsymbol B\to0}\frac{\partial E}{\partial\boldsymbol B}=\frac{e}{m}\boldsymbol S\tag{8}
$$
To look for quantum corrections to this relation, we note that loop corrections to the vertex function modify the Dirac equation into
$$
\left(i\not\partial+e\not A-m+\frac{ea_e}{2m}F^{\mu\nu}S_{\mu\nu}\right)\psi=0\tag{9}
$$
where $a_e\equiv F_2(0)$ is the electron anomaly (and $F_2(q^2)$ is the Pauli Form factor), and $S^{\mu\nu}\equiv \frac{i}{4}[\gamma^\mu,\gamma^\nu]$. By using the exact same reasoning as before, we can show that this loop-corrected equation leads to
$$
E^2=\boldsymbol p^2+m^2-2e(1+a_e)\boldsymbol S\cdot\boldsymbol B+\cdots\tag{10}
$$
and therefore
$$
\boldsymbol\mu=\frac{e}{m}(1+a_e)\boldsymbol S\tag{11}
$$
which shows that the gyromagnetic ratio of the electron is $g=2(1+a_e)$. In any case, the key identity to prove the relation above is
$$
S^{ij}=\frac{i}{4}[\gamma^i,\gamma^j]=\frac12\varepsilon^{ijk}\begin{pmatrix}\sigma^k&0\\0&\sigma^k\end{pmatrix}\tag{12}
$$
which makes it very clear why $S^{\mu\nu}$ is closely related to the magnetic moment: the matrices $S^{ij}$ are just the spin part of the angular momentum of the electron. The Levi-Civita symbol $\varepsilon^{ijk}$, when contracted with $F^{\mu\nu}$, produces the magnetic field $\boldsymbol B$ (why?), and therefore a term $S^{\mu\nu}F_{\mu\nu}$ is essentially a coupling of $\boldsymbol S$ to $\boldsymbol B$.
I leave it as an exercise to fill in the details.