Crystal field in diamond

The crystal field effect occurs in ionic crystals and causes a splitting of the magnetic quantum levels of the cation. The magnitude of the splitting may be roughly computed by obtaining the potential at the location of the cation due to the surrounding nearest-neighbor anions and using degenerate perturbation theory. My understanding is at the level of "Optical Properties of Solids" by Mark Fox (Amazon link)

Diamond is a covalent crystal. But, in case of the $NV^{-}$ center, it is said that the $m_s = \pm{1}$ levels are split from the $m_s = 0$ level because of the crystal field.

My question is: What is the origin of this crystal field? There are no "ions" in diamond - so, how do I understand this effect?

Crystal field is not related to ionicity. It is any perturbation which has the symmetry lower than full rotational symmetry due to symmetry of crystal lattice. In diamond case, the symmetry is quite high $O_h$, but still lower than $SO(3)$.

In the case of NV centers however, the story is even more tricky. It is not the point object: it is a pair of defects located nearby. So, in this case there will be a definite axis and the symmetry reduction is enough to get the splitting between 0 and $\pm 1$.

One important thing to remember is that since the NV center is a defect, the way the electrons bond near it can be very different from the way they bond in the rest of the lattice.

The negatively charged nitrogen-vacancy center defect in diamond's electronic states are understood to be composed of six electrons. Five of these are from unpaired electrons from the nearest neighbor nitrogen and carbon bonds, and an additional electron is thought to come from somewhere else in the lattice. Group theory applied to quantum mechanics can determine what the allowed total electronic states are based on symmetry. In other words, the allowed shapes/symmetries of the electron clouds are determined by the symmetry of the defect and surrounding crystal environment.

Like an atom, these states have different energies. The six electrons fill them roughly from the bottom up, and the electrons can be unpaired or paired with other electrons of opposite spin. In the case of the NV-, there are two unpaired electrons in its ground state, one in a state labeled "ex" and another in "ey" (these names originate from group theory notation and aren't important for this answer).

The origin of the crystal field splitting comes from a strong spin-spin interaction between these two states. The interaction between the two spins has a 1/r^3 dependence, meaning the closer the electrons are to each other, the stronger this repelling force is, and it turns out that the strength of this repelling force between the electrons in ex and ey in the ground state is about ~2.87 GHz.

Some great references on both experimental and theoretical results of diamond NV centers relevant to this question are:

1) J. Loubser and J. A. van Wyk, Reports on Progress in Physics 41, 1201 (1978).

2) Xing-Fei He, Neil B. Manson, and Peter T. H. Fisk Phys. Rev. B 47, 8816 (1993)

3) Theory of the ground-state spin of the NV− center in diamond, M. W. Doherty, F. Dolde, H. Fedder, F. Jelezko, J. Wrachtrup, N. B. Manson, and L. C. L. Hollenberg, Phys. Rev. B 85, 205203 (2012)