I think it's neater to express the various directions in terms of unit vectors. We can convert to angles at the end. So say that $\hat n_1$ and $\hat n_2$ are unit vectors perpendicular to the axes of the coils, and let $\hat r$ be a unit vector pointing from the center of coil 1 to the center of coil 2. Since the coils are separated by a distance that's large compared to their diameters, I'll assume that each one can be approximated as a magnetic dipole.
The magnetic field produced by coil 1, at the location of coil 2, is
$$
\vec B \propto {1\over D^3}[3(\hat n_1\cdot\hat r)\hat r-\hat n_1]
$$
(This is a standard formula for a dipole field. See a textbook such as Griffiths. Since you graciously allowed an arbitrary constant that had to be measured, I'm writing proportionalities and ignoring boring constants).
The mutual inductance is determined by the flux through coil 2 due to this field. Because of the large separation, we can assume the field is approximately constant at the coil's location, so this is just $\vec B\cdot\vec A$. So the flux is just proportional to the component of $\vec B$ that's normal to the second coil. That is,
$$
\Phi\propto \vec B\cdot\hat n_2\propto {1\over D^3}[3(\hat n_1\cdot\hat r)(\hat n_2\cdot \hat r)-\hat n_1\cdot\hat n_2].
$$
Since the mutual inductance is proportional to the flux, this is the answer you're looking for. To express it in terms of the angles, I guess we have $\hat n_1\cdot\hat r=\cos a$ and $\hat n_2\cdot\hat r=\cos b$. The dot product $\hat n_1\cdot\hat n_2=\cos c$, where $c$ is the angle between the axes of the coils.
So I think the final answer is
$$
k D^{-3}[3\cos a\cos b-\cos c].
$$
If all of the various axes lie in a plane, then I guess $c=a-b$, in which case the angular dependence is something like $2\cos a\cos b-\sin a\sin b$, but if things can swivel around in all three dimensions, then $c$ is not uniquely determined by the other two.