Ruling out
It's not to do with the speed that the air is coming out of your mouth. You can test this by blowing really lightly with your mouth almost closed, and then by blowing as hard as possible with your mouth wide open (so that when your mouth is closed the air flow is less fast than with the mouth open): it's still warm with your mouth wide open and cold with your mouth almost closed.
My theory
I suspect that because of the small gap when your mouth is nearly closed and the curved lips and then also different surrounding air pressures, all these would cause most of the air that's flowing from your mouth to spread out, which means that less molecules (albeit still a lot) are hitting your hand than when blowing with your mouth wide open.
Because there are less molecules hitting your hand at any one time (also less densely) you inevitably have far less energy hitting your hand as well, which in turn means less heat. Because there are more molecules hitting your hand at any one time (also more densely), there is more energy hitting your hand as well, which in turn means more heat.
Summary
So to sum up what I have just suggested, which seems to me a very liable suggestion, the difference in temperature on your hand is to do with the fact that there's a difference in the number of and density of molecules hitting your hand at any one time.
Note: I may have seemed quite ambiguous when talking about the "density" of the air hitting your hands. When I say "more dense" I mean that there's more $air/mm^2$ hitting your hand.