The standard argument about the vanishing of the magnetic field outside an infinite solenoid pertains to the longitudinal component, parallel to the solenoid axis. But there must be a non-zero toroidal field. However tightly the solenoid is wound, there is a net current in the longitudinal direction. Thus, by Ampere's law, there is a toroidal $B$ field whose magnitude dies off as $1/r$. Why is this field never mentioned? Or am I wrong?
I'm fairly sure you're correct, and in fact, it's mentioned on Wikipedia:
Of course, if the solenoid is constructed as a wire spiral (as often done in practice), then it emanates an outside field the same way as a single wire, due to the current flowing overall down the length of the solenoid.