Well, Active Noise Control has been around for quite some time and there's a lot of applications developed based on those principles.
Apparently you cannot effectively cancel out one source if the main source and the cancelling source do not coincide in space. This, of course, has practical limitations, but it also depends on the wavelengths of interest. When the wavelength is quite big, you can achieve destructive interference in larger areas. If the distance between the two sources happens to be well shorter than half a wavelength, then if you consider omnidirectional radiation, then they will always maintain their phase relationship, which means that you could effectively cancel the main source quite effectively. Of course, this has a limited frequency range and as already mentioned it depends on the wavelength and the distance between the two sources.
Now, considering the fact that you have in mind using only one speaker which will be very far from some of the sources means that the frequency range for which you could achieve attenuation would be quite small. In addition to that, this would produce an increase in pressure in other positions. This is what makes active noise control on three dimensional systems (such as rooms) quite impractical. If you could introduce loudspeakers in each desk, then you could possibly achieve attenuation on a quite large area for some low frequencies. This would end up being a quite complex system though.
In addition to that, you would have to consider the fact that lowering low-frequency noise could potentially increase speech intelligibility, increasing the annoyance of other people talking!
I strongly apologize for not providing in-text references. If you are interested in learning more about Active Noise Control you can start with the Active Control of Noise and Vibration books by Hansen and regarding annoyance from increased speech intelligibility, you can find more information in the Effects of Noise Reduction on Speech Intelligibility, Perceived Listening Effort, and Personal Preference in Hearing-Impaired Listeners article by Brons et al. and the Effects of Interior Aircraft Noise on Speech Intelligibility and Annoyoance paper by Pearsons and Bennett.