The most straightforward explanation is that the liquid, being a better thermal conductor than air (by a factor of about 20), is drawing heat from your finger faster.

Modeling this multilayer heat transfer process isn't trivial, but we can refer to [Fourier's law](https://en.wikipedia.org/wiki/Thermal_conduction#Fourier's_law) and conclude that for any given temperature difference between your fingertip and inanimate objects at room temperature, the heat flux over a given distance increases with increasing thermal conductivity. The container material is the same in both cases, so we look to the material properties of the medium directly behind it.

The same effect occurs with heating; putting your hand in a 100°C oven isn't immediately hazardous, but contact with 100°C water would burn you.