After all, they are a (self-sustaining) perturbation of the same field, like sound waves or water waves are "energy flow" (except these ones experience dissipation). And how can our eyes be so clever to perfectly sort and recognize objects if the air is "polluted" with all kinds of photons bouncing all around?
You just raised a question about a very important topic, the distinction between interference and interaction. A lot of answers on this site mention interference in connection to the double slit experiment. And you see other phrases like "photons do not interact with each other". I think this needs a little clarification:
- Interference, you can see this from the double slit experiment, done by shooting single photons at a time. Emphasis on single photons. What interferes with what? You just shot a single photon. The pattern arises only if you repeat the experiment, and shoot many photons after each other. The boundary conditions are all the same, and each photon that is shot from the same setup laser, the interference will show up, showing an interference between the photons that were actually shot after each other.
- interaction, this is about the vision question in your example. the photons bouncing off objects do not interfere with each other (visible wavelength and energy level in your example), to the first order. Photon can and do interact, but you need much higher energy levels, and that is called nonlinear optics. We are lucky that at the visible wavelength energy levels there is linear optics, and no photon-photon interaction, because otherwise we would not be able to see.
The four electromagnetic vertices make the contribution so small , it can be ignored for visible light frequencies. The electromagnetic spectrum has higher energy photons though, up to gamma rays, and the probability of photons scattering goes up with energy
So the answer to your question is, that photons do interact, but that becomes an apparent phenomenon only at high energy levels, much higher then the energy of visible photons, thus we are able to see.
Photons do interfere, there are places where you can see the classical interference patterns like in the double slit experiment (or every interferometer) and some places you can see quantum interference (e.g. Hong Ou Mandel experiment).
The "sorting" of photons is an outcome of the lens in our eye, sorting photons coming from different directions to different places on our retina. The sorting by color is due to the different wavelength sensitivity of the detectors in each "pixel" on our retina (read more about the RGB cones)
Photons of different energy have different wavelengths. When they interfere with each other it isn't done in a linear fashion. Our sensors in our eyes can understand only a few frequencies of light.And the information of each wave is not lost in the collection of waves "polluting your eye".
Your question is correct, photons do not really interfere. The DSE taught at the high school level is a convenient theory and it also works well mathematically but 2 photons cancelling is a violation of conservation of energy. In university in quantum optics courses deeper explanations are provided.
Think of 2 tsunamis one from Japan and the other from USA, starting with opposite phase .... when they meet (at say Hawaii) they cancel and Hawaii is saved ... but a second later the waves emerge again and continue on their way to Japan and USA, the energy was only stored temporarily in the elasticity of the water! The energy will only be absorbed when the wave crashes on the land. For photons we can never really observe the field directly ... we can only see a photon when our eye or camera absorbs it. We assume the photons are interfering in the EM field .... it makes sense .... but every photon is created by an atom and eventually absorbed by an atom.
The waves associated with a single photon can interfere with each other (and contribute to producing an interference pattern). Different photons in a laser beam (all of which have the same wavelength and phase) can also interfere with each other (making holograms possible). Photons from an ordinary light source may have many different wavelengths and no fixed phase relationship. Any interference effects would be fleeting and vary from point to point.
Photons are particles and not to be confused with electromagnetic waves or wave packages. They do not interfere. EM waves do interfere. The EM interference pattern, more precisely$^*$ the value of $E^2$ at a position, gives the probability to detect a photon at that position.
$^*$ This assumes the photon is detected by an electric dipole transition. For a magnetic dipole transition $B^2$ is the relevant quantity.