Why do the spins follow the bar of the magnet when you turn it in your hand? So we know from experience that when we turn a magnet in our hand, the magnetic dipole doesn't lag behind. So if the magnet (and its dipole) is pointing down, when we push on the material of the magnet and rotate by 90 degrees, the dipole moment follows along. That this is even possible isn't obvious from quantum mechanics.
From quantum mechanics, we know that the only way to make spins change orientation, is from hamiltonians that are of the form $\hat H = A_i \hat S_i$, that is some linear combination of spin operators. This ignores the case when the $A_i$ are operators of another system in which case it just gets entangled.
Now from physical considerations, $\hat H = -g\mu_B \vec B \cdot \vec S$ usually. That is I need a magnetic field to rotate the spins. When I turn the dipole moment in my hand, as far as I can tell there isn't any magnetic field that I'm making. I only apply normal forces. So my hand can't be whats rotating the spins!
Question

So the question is what is the cause of the magnetic field that
  rotates the spins of the magnet that cause it to follow along the
  physical orientation of the bar. It must be a magnetic field as you
  can't grab an electron by some handles and torque it any other way.

The only things I can think of is maybe when I rotate the bar magnet I generate a current of protons from the electrons perspective that gives a magnetic field that might make them rotate. But it’s not obvious that that would give exactly the right rate. Especially since the g factor of an electron is 2. (This g factor of two makes spin degrees of freedom rotate twice as fast as orbital degrees of freedom).
Another possibility, is maybe the rotating macroscopic dipole moment itself can ensure that one dipole moment follows along by the adiabatic theorem but that assumes that somehow all the spins rotate to begin with. 
 A: 
I know that spins only rotate due to magnetic fields

I'm not 100% sure what you're saying here, but at face value, no.
The magnetic moment of the bar is the sum of all the individual moments of the objects within it. Those moments have a certain direction relative to the objects. Those objects are being held in a particular arrangement due to the mechanical structure of the bar. When you turn the bar they retain their original orientations relative to each other, so the moment remains aligned with the bar as it was before.
Is this what you are asking?

It must be a magnetic field as you can't grab an electron by some
  handles and torque it any other way.

Ahhh, ok!
There is indeed a way to grab an electron and torque it. That's because there is an interdependence between the spins and the distribution of the electric charge, as Pieter noted, the spin-orbit interaction. In ferromagnetic materials, where the underlying structure of the material results in a repeating electric structure, the spins of the electrons align with that field. So when you turn the magnet, their spins go with it.
I think this article will get you going.
A: There is the spin-orbit interaction which couples the electron magnetic moment to its orbital moment. This gives rise to magnetocrystalline anisotropy: it is easier to magnetize the material along some crystal directions than along others. The energy differences are small (micro-eV per atom), but that is how spin direction is coupled to the atomic structure.
