Three things to consider:
- The temperatur $T$ of the glass,
- the thermal conductivity $\kappa$ of the glass, and
- the fact that you are actually comparing conduction from skin-to-glass with convection from skin-to-air, which might not be a fair comparison.
To no. 1., remember that your hand is much warmer than the glass. The glass temperature is not constant though, see below.
To no. 2., consider e.g. steel and wood. At the same temperature, what feels hottest? The steel! Why? Because the thermal conductivity is much bigger in steel than in wood. So even thought there is the same temperature difference between the materials (steel or wood) and your hand, which will result in heat transfer into your hand (so your hand heats up and it feels warm), this rate of heat transfer (that is, how fast your hand is heated up) is much higher in the steel.
But what really should explain it is no. 3. You don't loose much heat from your skin to the surrounding air inside your house. Heat transfer from you-to-air is due to convection. Now if you touch the glass, then conduction of heat starts, which is much more effective, and much more heat at a much higher rate is transfered from your skin. (And loosing heat fast is equal to feeling cold, since the molecules on the edge of your skin are cooled down faster than they can be reheated by your body.)
The glass temperature
The glass does not have just one temperature. It has a temperatur distribution throughout it's depth. See this illustration:
The yellow block is a cross-section of the glass. We have inside to the left and outside to the right.
The air on the inside has a certain constant temperature $T_{\infty,1}$ (we call it constant because there is so much air; even though the window cools down the air a bit this has no sensible effect). Near the glass the air is cooled down, and you see the temperature of the air dropping to $T_{s,1}$.
The glass then has a constant decreasing temperature inside itself. This temperature varies linearly from the temperature of the inside-surface $T_{s,1}$ to the temperature of the outside-surface $T_{s,2}$.
And the cooler ouside air is then a bit warmer with temperature $T_{s,2}$ near the glass but reaches it's constant outside-temperature $T_{\infty,2}$ as you move further out.
This not-temperature-constant boundary layer of the air near the surface of the glass is very small.
From all this we can also now see that the glass feels colder, if you touch it from the outside than from the inside!
But since it is very thin the difference might not be that big. (If you try this out, remember that it is a different story for two-layer glass windows. This can be modelled similarly and there will be a big difference in temperature distribution between the inner and outer layer of glass. The gab in between is filled with air, so here we have convection again - but in still air convection is very week and the air acts as a proper thermal isolator.)