Why does motion help you balance on ice skates? It's almost impossible to balance on a single ice skate if you're standing still.  But give yourself just a little forward motion—it doesn't take very much—and it suddenly becomes easy.  You can stand there on one leg and glide effortlessly half way across the rink.  Friction will cause you to gradually slow down, and it's only when you've slowed almost to a complete stop that balancing becomes hard again.  Why?
This seems suspiciously similar to the question of why it's easier to balance on a moving bicycle.  But the standard answer to that involves the angular momentum of the wheels.  There's nothing rotating here.  In fact, other than the very slight deceleration from friction, moving should make no difference.  In both cases, you're standing still in a nearly inertial reference frame.
Does that slight deceleration somehow matter?  Or does the interaction between the ice and a moving skate somehow help you to balance?
 A: I assume you are talking about skates for figure skating (and not about skates for speed skating). 
Figure skates are not straight. Along the length of the foot there is a curve, in the vertical plane.
As you are gliding over the ice your sense of balance feels it when your center of mass is no longer precisely above your contact point. You correct by pointing the skate ever so slightly in the right direction to move your point of contact back underneath your center of mass. Pointing ever so slightly is enough; the motion of the gliding does the rest 
It may even be that you don't need to make that adjustment actively. When the shoe with the skate leans to the right I assume the skate itself will tend to follow a slight curve to the right, automatically bringing your point of contact back underneath your center of mass.
(I have some limited experience with speed skates, that's why I'm offering guesses rather than speaking from direct experience.) 
A: This simple question requires a complex answer.
The bike comparison is misleading. A bike has one more degree of freedom (or let's say one more degree of control, the handlebar). You don't have it with an ice blade or with inline skates.
I really believe that for ice skating, and rollerblading too, there are two concurring actions that keep balance. One is a self-balancing mechanism where the skater is an "unbrained" passive system. The other one is the voluntary correction - a brain-controlled negative feedback loop - for off balance situations. This second mechanism has already been pointed out, but not the first one.
If you have some ice skating experience you'll agree that a fast spiral on a large curve is self balancing. Once correctly entered, the track develops by itself without any need for correction.
The reasons for such a good performance are manifold: (1) blades are manufactured by experience for having their best behaviour on ice. They have rocker, two edges, and a hollow profile. This builds a small amount of friction when the blade steers left or right. It is necessary to slowly dump any angular momentum on the vertical axis. Otherway it would be very difficult - or even impossible - to control it. (2) skater's height (and his/her center of mass height above the ground, about 100 cm) is several times bigger than the blade length and its effective - few centimetres - footprint on the ice (3) the normal friction of a blade, along its travelling direction, is very small but not completely negligible. In fact the horizontal deceleration is around 1 hundreth of the gravitational acceleration (9.8 m/s2) (4) Said that, the center of mass of a skater - during a passive glide - always stays behind the center point of contact of the blade with the ice. Say 1/100 * 100 cm = 1 cm. In other words the skater's body "lags" just a bit her/his foot trace. (5) This fact develops a torque, on any curved path, that keeps the skate steering just the right amount.
A: In motion, the skate easily tracks left and right to find balance. At rest, the skate resists side to side motion. The skater awkwardly pivots over the fixed skate waving arms.  Fore and aft skate motion however, is all too easy, usually the beginning skater's downfall.  Any forward motion of the skate while the body is stationary results in an unsupported condition. 
