How does a wave become a color? I am a mathematician and not a physicist, so please be gentle to me.
The best explanation I found so far about light waves was here: https://youtu.be/7eutept5h0Q
If I got it right, then light is a particle without mass which has magnetic and electric energy and this energy is oscillating all the time. When one spots this particle, one sees a color which depends on the frequency of the oscillating energy. Is that correct?
Also every color in the below diagram may be observed by a single particle.

All other colors that don't appear in the diagram (like white color) have to be a collection of light particles of differed frequencies which will appear to us as a single color?
 A: You show above .the visible light frequencies, which are the colors of the rainbow. Visible light is composed out of elementary particles called photons whose energy is given by $E=hν$, which are mathematically described by wavefunctions that are solutions of the quantized Maxwell equations. A quantum superposition of zillions of photons build up the classical electromagnetic wave. Visible frequencies are a small part of the spectrum.. 
But there is another aspect of color, called color perception, what hits the retina in our eye and the brain assigns a color to it.

So

If I got it right, then light is a particle

No, light is a superposition of quantum mechanical zero mass photons

which has magnetic and electric energy and this energy is oscillating all the time. 

Only in the superposed  composed of  zillions of photons classical wave. The particle/photon has a fixed energy and is neutral, the electric and magnetic fields are built up by the superposition of zillions of photons. The photon itself, just has energy, and spin and momentum. The electric and magnetic fields are in the complex wavefunction whose complex conjugate squared gives the probability of finding a photon at x,y,z. See this experiment. 

So seeing a color does not define uniquely the frequency.

that is correct.
For 
 comments:
To answer  whether single photons/ quanat  exist, here is the double slit experiment single photon at a time


Single-photon camera recording of photons from a double slit illuminated by very weak laser light. Left to right: single frame, superposition of 200, 1’000, and 500’000 frames.

It is a quantum of the light, and the frequency can be extracted from using slit width, distance  and the accumulated interference pattern of the classical manifestation. After all the wave nature is in the probability distribution displayed nicely here on the right.
Is it a wave packet hitting singly and seemingly randomly on the left? each point could be modeled with a narrow wavepacket consistent with the frequency.
For absorption and emission  sectra , the lines have a width, and the wavepacket if within the frequency width will act as one quantum, imo. 
A: The color is not related to frequency in 1 to 1 relationship. Frequency is a well defined, measurable property of the light. Color is a sensation which depends on the frequencies of the light hitting the retina but not in a simple way. You can get the same sensation (color) either by light with a narrow range of frequency (for example  green produced by a laser pointer) or by mixing light with different frequencies (like mixing blue and yellow may result in the sensation of green too). So yes, the color depends on the frequency range of the light. It is more common that light hitting the eye is a mixture of waves with a wide range of frequencies so this is how we get the many shades of colors. The sensation of color may depend also on the environment, the brightness of the object and the background.     
A: Put simply, light is an oscillation in the electro-magnetic field, with a certain frequency (and other properties, but they are irrelevant to the question).
The photo-receptors in the human eye are sensitive to light with frequencies that correspond to wavelengths of ~380nm to ~780nm (wavelength is inversely proportional to frequency) - this is called the visible range.
every frequency or combination of frequencies in the visible range are perceived as a color by our brain. Any frequency outside the visible are, well, invisible to our eyes.
