Overblowing is a phenomenon that exists in all wind instruments. The details of the physics are different from one instrument to the next, but there is a broad similarity, which is that it's the result of a nonlinear interaction between the air column and whatever is driving the air column.
The recorder is in fact one of the simpler examples to understand. The mechanism that drives the air column is called an edge tone. The mouthpiece of the recorder contains a knife edge. The stream of air encounters the knife edge, but doesn't split smoothly onto the two sides. Instead, it forms a vortex which carries the energy to one side of the edge. However, a feedback process then causes this pattern of flow to deflect until it flips to the other side of the edge. So this is a highly nonlinear system. It's binary. Air either flows to one side of the edge or the other, and if we label the two states 0 and 1, we get a pattern over time that looks like 0000111100001111... You could graph it as (approximately) a square wave.
When this edge-tone system is coupled to an air column, it's forced to accomodate its frequency to the resonant frequencies of the column. For example, if you imagine a pulse emitted from the edge, this pulse then travels down the tube, is partially reflected at the open end, returns, and slaps against the air in the edge-tone system, influencing its evolution. There is a tendency for the edge-tone system's vibrations to become locked in to one of the resonant frequencies of the column. In overblowing, the pattern switches from 0000111100001111... to 00110011... The square wave doubles its frequency from the fundamental frequency $f_0$ to the first harmonic $2f_0$.
The original square wave contained Fourier components $f_0$, $2f_0$, $3f_0$, ... The new one contains $2f_0$, $4f_0$, $6f_0$, ... As you observed on the oscilloscope, $f_0$ is absent from the overblown spectrum. The ear's sensation of pitch is based on the frequency of the fundamental, so we hear a jump in pitch.
Bamboo flutes and whistles also use edge tones, so exactly the same analysis applies. I think the original classic work on this was an analysis of organ-pipe acoustics in a German-language paper by Cremer and Ising. In general, the edge-tone system could be replaced by a reed, lip reed (as in brass instruments), or air reed (flute). There can be overblowing at the octave, or, in instruments such as the clarinet that have asymmetric boundary conditions, at an octave plus a fifth (i.e., a factor of 1.5 in frequency). On the saxophone, for example, a skilled player using a stiff reed can overblow to frequencies corresponding to several higher harmonics beyond the first.
Cremer and Ising, "Die selbsterregten Schwingungen von Orgel," Acustica 19 (1967) 143.
Fletcher, "Sound production by organ flue pipes," J Acoust Soc Am 60 (1976) 926, http://www.ausgo.unsw.edu.au/music/people/publications/Fletcher1976.pdf
Backus, The acoustical foundations of music, Norton, 1969, pp. 184-186.