If you are considering human vision there is a definite (and surprisingly small) number of distinguishable colors.
This is known as a MacAdamMacAdam diagram and shows a region around a single color, on a chromaticity diagramplot, that is indistinguishable from the color at the center.
The total number of colors would be the number of ellipses needed to completely fill the color space. Obviously this depends on the individual's age, sex, lighting, etc