Why is it so difficult to pull an immersed mug out of the water? Suppose I put a mug in the water and then tilt it so that the water gets inside. It then floats due to buoyant force. But if I try to pull it out then a lot of force is needed. That's due to suction? What exactly happens at the moment when the force I apply is enough to pull the mug out? And what causes the loud pop sound when that happens?
 A: If the mug is completely full of water, the force required to lift it out is the same whether the mug is right-side-up or upside-down.  When the mug is just ready to break free of the surface, that force is equal to the weight of the mug plus the weight of the water to fill it.  That's because the mug is full of water in either case.  
In the upside-down case, the water all flows out when the mug breaks free of the surface.  At the same time, air rushes into the mug through the narrow gap that has opened between the rim of the mug and the surface of the water.  That's what makes the sound you hear.
Why does the water stay inside the mug until it breaks free of the surface? Because there's no way for air to get inside, so if the water level were to drop, it would leave a vacuum above it.  Atmospheric pressure keeps this from happening. If the mug were tall enough, eventually the weight of the water in the mug would overcome the atmospheric pressure.  At sea level, as you lifted your giant mug past 10 meters above the water surface, the inside water level would stop rising with the mug, and an evacuated space would appear above it.  The force required to lift such a mug to the point where it breaks free is still equal to the weight of the mug plus the weight of the water inside (though now it is not entirely full of water).  But in this case, the force is also equal to the weight of the mug plus the weight of the column of air above it, as described by Martin in his answer (the bottom of the mug feels atmospheric pressure from the outside, and no pressure from the inside).
A: There entire weight of the atmosphere is pushing down on the base of the mug, normally the same atmosphere would flow around the sides and into the mug balancing this out. If the water is stopping the air flowing in then you are trying to lift a cylinder of air the size of the bottom of the mug and going to the top of the atmosphere.
The atmosphere weighs about 15 pounds/inch $^2$, or 10 tons / meter$^2$ 
