until its inner pressure equals to the atmospheric pressure
The inference that the balloon is not growing (or shrinking) because the pressure is the same is not correct.
The balloon is not growing because the effective force pushing the balloon out from inside is the same as the effective force pushing the balloon in from outside.
The force pushing outwards is indeed due to the pressure of the air inside the balloon.
But the force trying to collapse the balloon is the pressure of the air on the outside (atmospheric pressure) plus the elastics potential of the balloon trying to return to its original size and shape.
So, to counteract this additional force the pressure inside the balloon has to be higher than the air pressure outside the balloon.