Your question can prompt answers in two seemingly separate directions. One direction considers the gas molecules as individual systems onto themselves. The other considers the entire gas as a system. Let's try to tackle both properly.
Consider an idealized picture of the gas as being self-contained even as it expands. Call that self-contained "bubble" of gas the system. The external pressure on the system is zero when that bubble is surrounded by a perfect vacuum. The thermodynamic definition of mechanical work for an expanding system is $w_{mech} = -\int p_{ext} dV = 0$. This is free expansion of the system. We say the gas as a bulk system does no work (on the surroundings).
Now step inside the system. Stand at any point in the gravitational frame of "up" and "down". Stand on a gas particle and imagine that it moves "up" in the gravitational frame. That particle has increased its potential energy in the gravitational frame. The gain in potential must come at the expense of a loss in energy specific only to the particle. If the particle is an ideal gas, it has no vibrational or rotational modes (neglect also electronic and nuclear here). So, an ideal gas particle will slow down as it moves upward in a gravitational field. Alternatively, a "real" molecule has rotational and vibrational internal energy modes. The real molecule could move to a lower rotational or vibrational state as it moves up in gravitational potential energy.
Translation, rotation, and vibration are measures of the internal energy of the gas itself. None of the changes that increase the potential energy of an individual particle at the expense of decreasing its internal energy are defined as mechanical work. They are all defined as a change in the internal energy of the system. The system is not doing mechanical work. But the gravitational field is doing other work on the individual particles.
You can now speculate about how the placement of the gas bubble relative to "bottom" and "top" walls in a box might change the considerations. What happens for example to a bubble of ideal gas that starts at the "bottom" of a box and expands (freely) "upward" compared to a bubble of ideal gas that starts at the "top" of a box and expands (freely) "downward"? Does the gas in the former get cooler because the internal energy of the gas particles decreases while the gravitational energy increases? Does it go the other way for the gas bubble that expands "downward"?
The answer depends on whether the process is adiabatic or isothermal free expansion. In the former (a perfectly adiabatic free expansion), we can predict that an ideal gas bubble that expands "upward" should be cooler at its end state compared to one that expands "downward" in a gravitational field. The gas does no work in either case. But the gravitational field does work on the individual particles. How do we make this prediction?
The ideal gas particles that expand "upward" gain potential energy. They correspondingly loose kinetic energy. This is a loss in internal energy of the gas particles. For an ideal gas, internal energy is only dependent on temperature. Lower internal energy is lower temperature.
A complementary view is to use the full expansion of the first law
$$\Delta U = q + w_{mech} + w_{oth}$$
where $w_{oth}$ is the other work. With adiabatic free expansion of the gas in a gravitational field, $\Delta U = w_{oth}$. The gas molecules do work to move upward in a gravitational field. Therefore $w_{to}$ is negative. The internal energy of the particles decreases as they move "up hill". Assuming that this internal energy change is dissipated over the entire system, we conclude that the internal energy of the bulk gas decreases as well.