This is a question that has been bothering me for a while, and is not definitively cleared up by the references found in the links provided by Johanes.
Obviously at the expansion following the Big Bang everything is the stress energy tensor and is expanding as the universe expands. It is when the narrative of the appearance of particles comes about that extra hypothesis have to be taken into account. This is that once QCD, maxwells equations and the weak lagrangian are in effect, and particles are generated from the primordial soup, the mass systems formed keep their cohesiveness like corks floating on a river due to the strength of the interactions. Some references say that in the ADS calculations there is a small expansion for atoms, but nothing like the cosmological size.
In my opinion, until we have a theory of everything, and maybe strings et al will provide us with one, one cannot be dogmatic about this. After all, the wavelengths are expanding, which is why we know of the expanding universe, and electromagnetic waves are also matter and also governed by Maxwell's equations.
At the moment my answer is, probably you will have the same density until you hit the particle formation time, with a question mark.