Well, I guess it depends on what you mean by a sound wave. There are wave pulses that can be created in air that can absolutely do work on something else. There are Smoke Ring Launchers, which can create waves of air that could definitely, I don't know, knock over a playing card or something. But as for sound waves, like you said, you're dealing with air molecules being jostled around from an equilibrium position. As soon as the pressure waves cause the air particles to clumb together in some localized region, there are going to be areas of low pressure in the surrounding area that the air will rush back in to fill. So, the air is not going to have a lot of space through which to act on another body, or a lot of time to do so either for that matter.
Of course, sound waves CAN do work. You can break glass with sound. But it's usually not much.