Ultrasound is used to break kidney stones and "burn" brain tumors. So no surprise that it can damage body parts, including inner ear. The effects of ultrasound on human body are multiple. One of them is heating due to absorption of the mechanical energy in the tissue and conversion to heat. This is used in physiotherapy as a desirable effect but it can be damaging if too much power is absorbed. There is also a mechanical effect. Parts reached by ultrasound vibrate so if the amplitude is too high they may break. The effect is used in ultrasound cleaners as well in medical field, to break apart stones and blood clots. Again, too much can do bad things. For this reason there are strict rules on maximum power allowed in any ultrasound imaging system. These effects have nothing to do with the fact that our ear can or cannot hear the sound. In similar way, your eyes can get burned by microwaves even though you don't see them.
Richard Tingle mentions an interesting aspect of this problem.
I focused my answer above on the possibility of physical damage to the tissue due to ultrasound. In order to prevent these effectss, the limits for the maximum power used for ultrasound imaging is for example around 750 $mW/cm^2$ for sturdy organs like heart, liker, kidneys. For more sensitive areas like baby ultrasound or eyes the limits are much lower but stil of the order few (or tens of) $mW/cm^2$ . Now let's compare these with airborne ultrasound at 120dB level. In terms of intensity this means 1 $W/m^2$ or 0.1 $mW/cm^2$. And this is the same no matter if the sound is audible or not. The sensation we get depends on the frequency range but the physical intensity is the same.
This shows that the damage to the ears is of a different nature, and usually is a long term effect. One efect is an over-working of the sensitive hairs in the inner ear which in time can result of death of the cells in these hairs. The hairs are tuned to frequencies in the audible range. This does not mean that they do not vibrate when the frequency is off-resonance, just that the vibration amplitude is lower. Indeed, the limits established by the International Non-Ionizing Radiation Comitee for exposure to ultrasound are 110 dB for frequencies between 20 and 100 kHz (ultrasound) and just 75 dB for audible sound. This indicates that the effect of airborne ultrasound is less pronounced than that of sound. The difference between 110 dB and 75 dB is significant in terms of intensity. Hovewer the effect is here, as proved by many studies used by INRC to established their limits. The effects are not just possible hearing loss but also nausea, dizziness, headache and other things. So indeed 120 dB can be damaging even for ultrasound. You don't need to hear it to hurt you.