I would explain it by simply using the first law of thermodynamics:
$Q$ is heat added to the system, $W$ is work done by the system, $U$ is the internal energy.
Keep in mind that internal energy is closely bound to temperature, so a change $\Delta U$ also results in a temperature change (which is what we are talking about).
If you look at the gas as a system - a control mass - then when you open the valve and let the gas out, you are letting this compressed gas do work on the surrounding air by pushing it out of the way. So $W>0$.
When filling the cylinder with some apparatus that compresses the gas to put more in, there is done work on the gas, meaning that the gas does negative work on the apparatus. $W<0$
Assuming no heat exchange with the surroundings (let's say this happens too fast for any significant heat transfer to occour), $Q=0$.
Since $W \neq 0$, the first law of thermodynamics then tells you that there must be a change in internal energy.
And therefrom the temperature changes. Note that positive work (expansion) gives a decrease in internal energy, while negative work (compression) gives a rise.
This is experienced as heating or cooling by the observer that touches the cylinder because of the rapid heat conduction into the hand. Since the heat convection into surrounding air is much less effective, our assumption that $Q=0$ is fair as long as the cylinder doesn't touch any other good conducting material.