Clarification on temperature's relationship to container volume

My physics textbook says that if a container that contains some amount of gas is expanded then the temperature of the gas will decrease But isn't this only if the container is increased in volume because of the force the gas exerts?.

for example in the diagram below, if you pulled the piston outwards, so you were doing work on the piston, (not the gas) then the gas temperature would remain the same would it not?

We assume the gas is ideal and that $$n$$ is constant so $$pV/T=\mathrm{const}$$. If the volume increases all we can say is that the ratio $$p/T$$ decreases in the same proportion. We must be given some additional information to find how the temperature changes. Convince yourself that this must be the case. (For example, we could put the container over a Bunsen burner as we let the piston rise while applying some variable force. We should be able to make the temperature rise, remain constant, or even fall for particular choices of the applied force.)
In a situation like this the author is likely assuming that there is little heat exchange with the environment during the expansion. This is called an adiabatic process. In this case it can be shown that $$TV^{\gamma-1}=\mathrm{const}$$, where $$\gamma$$ is the ratio of specific heats. Thus, since $$\gamma=C_P/C_V>1$$, if the volume increases the temperature must decrease. Note that it doesn't matter how the volume increases, just that little heat was exchanged with the environment as it did so.