The most straightforward explanation is that the liquid, being a better thermal conductor than air (by a factor of about 20), is drawing heat from your finger faster.

Modeling this multilayer heat transfer process isn't trivial, but we can refer to Fourier's law and conclude that for any given temperature difference between your fingertip and inanimate objects at room temperature, the heat flux over a given distance increases with increasing thermal conductivity. The container material is the same in both cases, so we look to the material properties of the medium directly behind it.

The same effect occurs with heating; putting your hand in a 100°C oven isn't immediately hazardous, but contact with 100°C water would burn you.

The most straightforward explanation is that the liquid, being a better thermal conductor than air (by a factor of about 20), with a greater volumetric heat capacity (by a factor of about 3000), is drawing heat from your finger faster, and stays near its original temperature longer. (Thank you to Puk for pointing out the latter factor.)

Modeling this multilayer heat transfer process isn't trivial, but we can refer to Fourier's law and conclude that for any given temperature difference between your fingertip and inanimate objects at room temperature, the heat flux over a given distance increases with increasing thermal conductivity. In addition, as mediated by the heat capacity, a certain volume of water heats up much less than air for a given flux. The container material is the same in both cases, so we look to the material properties of the medium directly behind it.

The same effect occurs with heating; putting your hand in a 100°C oven isn't immediately hazardous, but contact with 100°C water would burn you.