The type of microscope and illumination scheme are not described. However, in general the microscope produces a real image down inside the tube just below the eyepiece. For attaching a camera, a separate port is sometimes provided such that the camera image sensor can be placed at this real image plane, or at a relayed real image plane. With an SLR type camera, however, it might be difficult to get the ground glass/ image plane down this low just by removing the eyepiece -- it depends on the microscope design.
By grossly adjusting the objective focus, it may be possible to move the nominal image plane up far enough beyond its usual position to reach the SLR image plane, but at that point both the microscope imaging and illumination optics may be operating very far from their design points, and this could possibly explain the observation of "fluff".
Absent an engineered camera adaptor, another approach which can produce usable results is to leave the eyepiece in place and place a camera (with its lens focussed at infinity) as close as possible to the eyepiece. The camera objective aperture should be kept fairly wide open.