The 6V in 6V battery is a label which gives an indication of the sort of voltage which might be obtained from such a battery.
For example if your ^ V battery was an lead-acid battery and it was fairly new and fully charged its voltage would be 6.3 volt and if older or partially discharges then it is more likely to be 6 V.
A 1.5 V alkaline battery at the start of its life would give 1.55 V. and a 1.2 V NiMH battery freshly charged would give 1.4 V but very soon this would drop to 1.2 V and then stay there for some time.
You will note that I have only used the term voltage without any qualification. The problem with real batteries is that even if they are of the same type and made by the same manufacturer they are not all they same.
If you measure the terminal potential difference with no current drain from a series of batteries of the same type you will not get the same value for all of them. Their age, how they have been used, their state of charge, the number of times they have been recharged, the temperature etc will affect the value.
Then if you start drawing current from a battery its characteristics will change and very much so as the current increases. For a start the terminal potential difference will drop due to a catch all parameter called internal resistance which for a battery is not really a constant.
Battery manufacturers produce copious data sheets which show extremely well how the electrical parameters of a battery change. They also give good advice as to what sort of battery to use in a particular application.
To illustrate the variability of the terminal voltage of a battery here are a couple of graphs for the Energizer NH15-2300 1.2 V NiMH rechargeable battery.