The question "Why does a slow-moving (or stationary) bicycle fall over?" is kind of a boring one. A stationary bicycle falls over because it is at an unstable equilibrium.
Specifically, a rigid body standing on a surface is at an equilibrium if its center of gravity is above the convex hull of its support (the points where it contacts the surface). If this is not the case, the object is at a disequilibrium, and gravity will cause it to fall over until it reaches a new equilibrium.
An object at equilibrium can be either stable or unstable (or, on the borderline between the two, neutrally stable), depending on whether slightly pushing the object (thereby placing it temporarily at a disequilibrium) will cause it to return to the original equilibrium, or to move to a different one. In particular, a rigid body at equilibrium on a surface is stable (against small perturbations) if and only if:
its center of gravity is above the interior of the support region, so that slightly moving the center of gravity will not move the object out of equilibrium; or
more generally, the shape of the contact surface is such that slightly tilting the object will move the contact area in the same direction as the center of gravity, but further. (An example of this case would be e.g. an ellipsoid lying on its side on a flat surface.)
(Note that this is not the fundamental definition of stability, which you can find on the Wikipedia page I linked to earlier, or in any decent mechanics textbook. However, it is, IMO, the most convenient stability criterion to apply here, despite its limited applicability to more complex systems.)
A stationary bicycle, standing upright, is approximately a rigid body standing on two points (where the wheels touch the ground). Thus, its support area is a narrow line connecting those points, and, even if the center of gravity of the bicycle is initially exactly above the line, even the slightest push sideways is enough to move it out of that equilibrium.
(A bicycle, of course, is not a completely rigid body, since it has moving parts. However, for static stability analysis, treating it as if it were rigid turns out to be a good enough approximation.)
Further, while the contact points do move slightly when the bicycle starts to tilt, they move much less than the center of gravity does (since the radius of the wheel tubes is much less than the distance of the center of gravity from the ground). Thus, as soon as the stationary bicycle tilts even slightly to one side, it will be at a disequilibrium, where the effect of gravity is to tilt it further away from the stable equilibrium, until it falls completely over and finds a new equilibrium with a wider support and a lower center of gravity.
In particular, observe that adding a third support point, e.g. in the form of a kick stand, can be enough to create a stable equilibrium at the position where all three points (both wheels and stand) touch the ground, and the center of gravity of the bike lies above the triangle formed by the support points.
As for the converse question, "Why doesn't a moving bicycle fall down, then?", that one has already been asked and answered here. In particular, the paper referenced in nibot's answer there provides a pretty definitive explanation.
Summarizing it briefly, there are multiple stabilizing effects (such as the gyroscope effect, the caster effect and the mass distribution of the bicycle — not to mention the active control provided by the person riding it, of course) that can stabilize a moving bicycle, but, as one of the authors of the paper describes at the end of this YouTube video, almost all of them act through a single mechanism:
To stay upright, a moving bicycle must turn in the direction it leans towards, and it must turn quickly enough that its support will move sideways faster than its center of gravity.
A slow-moving, or stationary, bicycle will not be able to do this, and therefore is not stable.
One notable exception is that the person riding the bicycle can exert a limited amount of direct control over their center of gravity by shifting their weight and, depending on the caster angle, by turning the handlebars even while stationary. It's not a very strong effect, though, which is why it's very difficult to balance on a stationary bicycle.
(Note that the standard technique for doing this, as described in the Wikipedia article above, essentially cheats a bit by using small amounts of back-and-forth movement. It's still difficult, but not nearly as difficult as it would be if you couldn't do that at all.)