As you may know, the friction is proportional to the normal force of an object or in this case the force of attraction between magnet and refrigerator. If your force is strong enough then the friction will be sufficient and the magnet will not slip (on earth the force of friction must exceed the mass of your magnet multiplied by 9.81 m/s).
If we assume your magnet weighs 0.010kg and is made of iron, while your refrigerator is steel then the coefficient of friction is 0.4 (Engineering Toolbox, Friction and Coefficients of Friction). The force exerted by the magnet on the steel is 5N.
Friction = 5 x 0.4 = 2N
Now we find that the Force down is equal to 0.010 x 9.81 ~ 0.1N
The magnet would stick