Let me set up the following convention for orientations, loosely based on aviation.
I take the bicycle wheel as reference.
Swivel: rotation around vertical axis
Roll: like roll of an airplane
Pitch: like pitching of an airplane.
Let the spinning of the bicycle wheel be designated as 'rolling'.
InquilineKea, in the picture I presume the bicycle wheel rolling, and I gather the bicycle wheel is swiveled, while preventing pitch of the bicycle wheel.
I gather you observe that as the bicycle wheel is swiveled the chair, with the bicycle-wheel-holding-person in it, tend to counter-swivel. (I'm assuming the entire weight of the bicycle wheel is supported by the person in the chair.)
As the bicycle wheel is swiveled you clearly need to prevent it from pitching. The torque to counter-act pitching tendency is a pitching torque. Clearly this pitching torque will not affect swivel of the chair.
So I concur with you: it seems to me this excludes the spin of the bicycle wheel from the explanation.
Try the same with non-spinning bicycle wheel. Swiveling the bicycle wheel will affect the chair at least somewhat, as the bicycle wheel has a considerable moment of inertia.
It would take a very strong person indeed to prevent any pitching of the bicycle wheel; the motion will be somewhat jerky. Possibly that jerkiness is muddling the effect.