What follows is very much the same as mgphys' answermgphys' answer, but I'm going to be pedantic about what I mean.
So imagine that
I manufacture a substantial body of areographene, and then carefully slice out of it a rectangular prism that is $h$ by $l$ by $w$ in size. This gives a volume for the material of $V = h l w$.
A put a analytic balance in a vacuum chamber and careful measure the mass $M$ of the test sample. Now I calculate a figure for the bulk density $\rho_\mathrm{bulk} = M/V$ which will be rather less than the density of air. This is the kind of measurement that people mean when they say that areogels and areographene are less dense than air.
However, if I retrieve my sample and look at it under a microscope I will see that is has a fine scaled structure in which filaments and sheets of graphene do not fill the volume of the material but instead form an open lattice work. So the real volume of graphene is not $V$ at all!
If I choose a liquid that will not damage the material I could carefully measure the volume $v$ actually occupied by the graphene using Archimedes' method.
With that I calculate $\rho_\mathrm{detail} = M/v$ which will be higher than the density of air. Now, because the air can move into the spaces between the filaments and sheets the volume of air displaced by the sample is $v$, not $V$ and the density we have to use to determine if it will float is $\rho_\mathrm{detail}$.
This is also why mgphys states that if we were to wrap up the sample with a thin, non-permeable membrane while in vacuum (and that membrane held, and the sample doesn't get crushed) it could float.