The surprising answer is that the stability of the modern bicycle has little or nothing to do with centrifugal force or gyroscopes or any of that.  Look up "bicycle stability" on Google.   Experiments show that the sloped angle of the front fork is very important, e.g. If the fork pointed backwards it is very difficult to stay upright at any speed.

At higher speeds a very slight turn of the handles moves the bicycle under the center of gravity of the rider quicker, so that the dynamical stability is improved.     As usual experiment corrects theory here.  

See the answer of Tristan at http://physics.stackexchange.com/questions/24/does-leaning-banking-help-cause-turning-on-a-bike/524#524 as well for an even better answer

See the comment of nibot below for a reference to an actual definitive article.