One can do a neat little experiment as described here and discover that atmospheric pressure is sufficiently strong to keep water up against its own hydrostatic pressure as long as there is a material to suppress the Rayleigh-Taylor instability (which is applicable fluids). But solids can similarly exert a net force per area downwards that could theoretically be supported by atmospheric pressure. Thus is there an analog of the Rayleigh-Taylor instability for solids? The reason fluids fall under gravity despite our atmosphere is the Rayleigh-Taylor instability, so what's causing solids to fall if they're sufficiently light?
The thing that holds the water up is the pressure (or lack thereof, a vacuum) between the water and the beaker.
That would also work for a solid that fills the beaker so close that no air can run around the edges.
US student machinists are often given a project to machine a piston and cylinder that fit together with only .001” (25 microns) between them. That’s close enough that very little air goes by. Hold it with the piston facing down, similar to the water, and the piston doesn’t fall out because no air can get between it and the end of the cylinder.