You are asking whether this popping sound is made by the difference in air pressure between the inside and outside of the balloon. Or the contraction of the material.
Actually it is none.
It is actually a wave that makes the popping sound. You would think it is a soundwave. Well it is that reaches your ears.
But the original cause is not the air.
It is the balloon material, that as it fractures, creates a phenomenon on its surface.
As per QM, the lattice molecules in the balloon material are held together by the EM force. The covalent bond that bonds the molecules together is created by electrons that are not shared by multiple atoms and multiple molecules. These valance electrons now exist as per QM around the whole molecule or group of molecules. Now these covalent bonds are constantly pushed against by the pressure of air inside. But what is exactly this pressure?
The pressure from inside is the vibrational energies of the air molecules as the gas tries to fill all available space volume.
Now after a certain time the material will not last, and there will be a fracture, caused by the brake in the covalent bond of some molecules in the lattice.
As this covalent bond brakes, air molecules start escaping between the lattice molecules of the balloon.
Now at this point the fracture in the lattice continues because the material's covalent bond around the molecules is not able to withstand the material's sideways tear force.
This is the start of a wave, inside the lattice of the balloon material.
As this wave travels, the lattice molecules form a wave and this wave formation is hitting the air molecules, transferring the waves into the air molecules structure.
A wave is formed now in the air molecules structure, and that is traveling to your ears, making you hear a popping sound.
As per the comment, I would like to use this site, as reference, I hope that is acceptable as reliable:
Now, Regarding the sound theme: The pressure inside (depends upon how much you blow) and outside (1 atm) the balloon creates a pressure difference in that area. This difference creates a longitudinal pressure wave (I'd say that as a small shock-wave) along with elastic energy of the balloon's material (while retaining to its original shape) which is perceived as sound.
Please see here: