What you are missing is that what is important for the overall colour of a galaxy is the colours of the constituent stars and their luminosities and how many exist in a particular evolutionary stage.
Comparatively rare, young high-mass stars can be both very blue and very luminous and have a large effect on the colour of a galaxy. But their lifetimes are counted in tens of millions of years, so ongoing star formation is required to replenish their numbers.
Most red giants are produced towards the ends of the lives of more numerous and longer lived lower mass stars. They are red (of course), but also luminous. There is an ongoing plentiful supply of ageing lower mass stars to produce them as they turn into white dwarfs.
White dwarfs are born hot, though they do not stay hot for that long - the period of time when they are both hot and luminous is very, very short and hardly any white dwarfs are found in that area of the Hertzsprung-Russell diagram. Most white dwarfs are moderately cool (like the Sun), but much less luminous. Thus even though they are being continuously produced and their numbers are growing, they also fade rapidly and contribute negligibly to the overall light from a galaxy.
