Yes it is correct.
According to the first law of thermodynamics:
$$∆U = q + w$$
(Here, $∆U$ is the change in internal energy, $q$ is the heat transfer between the system and the surrounding and $w$ is the work done on or by the gas in the system.)
During an adiabatic process, $q = 0$.
Now, $∆U = w = -P\,∆V$ (notice the negative sign here)
If work is done by the gas then there is loss in internal energy (or loss of the kinetic energy of the molecules) and thus cooling takes place. And if work in done on the gas then the internal energy increases and heating takes place which signifies increment in temperature.
And also, temperature is the measure of the average kinetic energy of the molecules of a system.