My professor's answer to this is "It will rise until the weight of air displaced equals the mass of the balloon". I am a bit confused by this. Wouldn't it depend on densities rather than weights and masses? I thought it will keep rising till the density of air is the same as helium. Can anyone point out where I am wrong?
You are correct that the balloon will keep rising until the density of the air equals the density of the helium in the balloon. But since $W=\rho g V$ the weight $W$ of the volume $V$ of air displaced by the equal volume of helium will also be the same as the weight of the helium (neglecting the weight of the balloon material) when the densities $\rho$ are equal .
Hope this helps.
Per Archimedes' principle, a body immersed in a fluid has a buoyant force equal to the weight of the fluid it displaces. As air density changes for various reasons, its weight per volume also changes. The helium's density may also change for various reasons. The balloon's volume may expand or contract with changing conditions. All of these things may change at different altitudes, temperatures, or other reasons, but Archimedes' principle will hold true in any set of conditions. If the total weight of the balloon and its contents are less than the weight of the volume of air it displaces it will still rise. By the way it would be more accurate for your professor to say "weight" of the balloon, rather than "mass" of the balloon, as weight can change slightly with altitude in Earth's non uniform gravitational field but an object's mass does not change.