Hydrogen wasn't really made at recombination, but in the hadron epoch, starting at one millisecond after Big Bang, and lasting about 1 second. Sure, it wasn't neutral hydrogen, but it was hydrogen nonetheless.
As you mention, neutral hydrogen was made at the epoch of recombination. But since the water doesn't contain free, neutral hydrogen atoms, this epoch isn't of much significance to the glass of water.
Water molecules was of course first formed after oxygen was created which, together with all other elements than hydrogen, helium, and lithium, was made in stars. The first stars came around a few hundred million years after the Big Bang, started enriching the Universe with heavier elements, and rather soon after, water molecules were present (Bialy et al. (2015))
Since you emphasize the word "all", if you want to be pedantic, not every single hydrogen nucleus was made during the hadron epoch, since you also have random fissions of helium nuclei in stars. They're just completely outnumbered by the number of fusions of hydrogen to helium.