Is there a physical reason for colors to be located in a very narrow band of the EM spectrum? The portion of the electromagnetic spectrum that is visible to humans are wavelengths between 380 and 750 nanometers.
I am aware that animals have different capacities than humans, but the EM spectrum where they see colors is very close, e.g. 300 - 590 for bees.
I am aware that some humans can see in quadrichromy, but what they see is actually two greens rather than one.
As all animals see around this visible light, it implies that it is in this EM band that the most information about matter can be gathered.
This band is therefore the best to distinguish between objects. Even color-blind people see shades of gray at these very wavelengths.
So it seems that matter has some special properties at the wavelengths of visible light that it doesn't at higher or lower frequencies.
It therefore seems plausible that there is a physcial phenomenon behind these, e.g. imperfections in compound matter could have sizes mostly corresponding to the visible wavelengths.
Is it actually the case?
Edit: added a few complementary questions to helps breakdown all the different aspects
Do most energy level transitions in matter of every day objects correspond quite precisely to the wavelenghts of visible light?
If no electronic transitions happened in the band of visible light, would we still be able to use this band to see? If no, what would be the most efficient ways to see?
 A: Apart from the "physics" explanations in the other answers, we must consider biology. To make an "eye" one needs some kind of lens (or, conceptually, a mirror -- evolution found the Galilean telescope, not the Newtonian, but maybe it could have) that focuses the incoming light, and receptors that get activated by these photons. There are (at least) three important factors for the selection of the wavelength:


*

*Focusing requires a surface that has the required shape with a high level of accuracy, less than the wavelength of the light that is to be detected. Thus, shorter wavelengths are harder to handle. Smallest cells are in the order of half a micrometer long; below we enter the realm of virus. A biological focusing apparatus would have trouble handling wavelengths shorter than the weakest UV.

*The receptor must receive enough energy to be activated. Long wavelengths imply less energy per photon, thus making activation harder. A long wavelength requires a larger eye, thus making the desired precision harder to achieve.

*Any complex structure consumes genetic resources; there are only so many genes that can fit in some chromosomes.
Thus, from an evolutionary point of view, there is some genetic pressure that means that successful species will be the ones that get one sort of eye (and not two or three kinds of eyes operating at distinct wavelengths), and that eye will be some kind of "optimal trade-off" between the need for enough incoming energy, and the ease of maintaining the proper focusing shape. It somehow makes sense that the optimal trade-off is located at wavelengths where light happens to be most plentiful on Earth, since more energy means that eyes can be smaller.
A: In addition to the answer by Martin, there is also the physical mechanism of detection of light.


*

*light with an ultraviolet wavelength can be absorbed to ionize an atom, triggering a chemical reaction; light in the visible region can only be absorbed to trigger excitation and require complex molecules with specific chemical properties to transform this excitation to an actual chemical reaction necessary for actual detection. (http://www.chemistry.wustl.edu/~edudev/LabTutorials/Vision/Vision.html) explains the biochemcial pathway whereby 1 photon absorption leads to a chemical signal cascade (discussion of which is probably outside the scope of this question)

*the near infrared window has almost no associated electronic excitations, as photons in this energy window do not have enough energy to excite an electron between any base level and any other level 

(Near-infrared spectrum of liquid ethanol, from http://en.wikipedia.org/wiki/Near-infrared_spectroscopy)


*

*with an environmental temperature of around 300 K, reception of wavelengths longer than 700 nm would be hard to measure using chemical processes, as anything chemically sensitive in that region would spontaneously react with thermal photons originating from its surroundings or even itself.



(from http://pveducation.org/pvcdrom/properties-of-sunlight/blackbody-radiation )
In conclusion, while very short wavelengths do interact with organic tissue, even short-ish wavelengths (UV) are still damaging (re:skin cancer caused by DNA polymerization and other chemical effects), the visible wavelengths are what we see, near-infrared have no easy ability to trigger chemical processes, and far infrared is what the detectors glow with as they are at 300 K, so would be too prone to spontaneous detection of non-imaging thermal photons.
The only reasonable wavelengths left with any resolution are those we actually see.
A: Mostly, you see things because they reflect light. They absorb some of it, which gives them their color, but you will also see them, if you shine infrared or ultraviolett light at them. So: Whatever light you shine at them, a large part of this light will be reflected and you can detect this light to "see" matter.
Your argumentation therefore seems backward. The most plausible idea is that most light on earth is of a given wavelength and therefore, most animals' eyes adapted to this wavelength. 

More precisely, have a look at the sun's spectrum: As you can see (yellow part), the radiation is most intense in the area of the visible light. This is due to the fact that the sun is a near ideal black body of the temperature of its surface. Now, the light that reaches the surface is not all of the light of the sun, since some wavelengths are blocked by the atmosphere (red remains), which is due to the fact that elements absorb certain levels of radiation. 
Now, light detection is more difficult, if there is less light (you can't see very well in the dark), hence it's easier to detect intensive light - thus it's a good idea to adjust your eyes to the area where light is most intensive.  
There are a few other aspects worth mentioning though:


*

*Note also that higher energy "light" can create other difficulties. Much of organic matter becomes transparent for gamma radiation (some even for x-rays - that's why tomographys works), which also means that it is much harder to detect x-rays with organic material, so it would be even harder to build an organic eye to "see" and make use of low intensities of gamma radiation. Still: with a good detector and enough intensive x-rays, I could probably also see a good picture of my surroundings.

*The same holds in the other direction: radio waves have very long wavelengths. A simple eye is not big enough to see them.
The upshot of all of this is: 


*

*Seeing the whole spectrum requires a much larger variety of detectors, one type of "eye" will simply not be enough. 

*Light on earth is most abundant in a narrow band of the electromagnetic spectrum
This does not explain why we only see a certain band of the electromagnetic spectrum, unless you want biological economy.
EDIT: So why do some animals see UV and none see IR light? Unlike I previously claimed, this seems to be more a biological problem: you'd probably need a very different "eye", similarly to what I hinted at when saying we need a larger variety of detectors: The only animals with really confirmed IR vision are snakes, who don't use their eyes to "see" IR light. On the other hand, all animals with confirmed UV senses use their eyes, they have just a slightly different window of visibility shifted to the ultraviolett, or simply another type of receptors (some birds apparently have up to five different color receptors, which also spread a larger band of wavelenghts).
I did not include a more complete survey of the biology - this is, after all, a question about the physics. See also Thomas' answer for a more complete argument of some biological arguments showing that it is probably not beneficial to have multiple eyes.
EDIT 2: There were some questions added for clarification, so let me try to answer those:

Do most energy level transitions in matter of every day objects correspond quite precisely to the wavelenghts of visible light?

Answer: No they don't. Let's have a look at the emission spectrum of hydrogen, the most abundant element in the universe and also very present on earth (albeit normally bound): Hydrogen spectrum and in particular this Wikipedia picture. We can see many lines, only a few of which are visible (four lines in the Balmer series). The NIST has a database of spectral lines for every element (see http://physics.nist.gov/PhysRefData/ASD/lines_form.html), where you can see that there is an abundancy of lines that are not visible. However, I don't know how probable all those transitions are. The Balmer lines for hydrogen are of course very probable transition.

If no electronic transitions happened in the band of visible light, would we still be able to use this band to see? If no, what would be the most efficient ways to see?

Assuming that we had a device to actually detect the light in these frequencies without using electron transitions (this is more a biophysical question and beyond my capabilities): We would be able to use this band, precisely because of what I said in my original answer: Most of the light we see is reflected sunlight, not absorbed and reemitted or just emitted light. Since sunlight is abundand precisely in the visible spectrum (and this has nothing to do with the emission spectra of atoms), we would see very well. However, colours will be problematic: Sunlight is white and the colours result from an absorption of certain parts of this light, while others are simply reflected. 
The absorption process is linked to the spectral lines, but I don't feel that I know enough to make this connection more precise. So it might be that the lack of any absorption in this part of the spectrum will make our world rather colourless - we'd see black and white.
A: EM radiation is aborbed in two ways:


*

*black body absorption

*electronic/vibrational/rotational excitation
All solids absorb due to the black body mechanism, (1),but all this does is convert the incoming radiation to heat and it's difficult to be precise about exactly where the incoming photon hit. Some very basic imaging can still be done, for example snakes use this method to get some information about the location of prey. However it's much less precise than imaging by the eye.
Method (2) is where the incoming photon changes the quantum state of a molecule. Rotational excitation is where it makes the molecule change angular momentum, vibrational excitation is where the molecule changes its vibrational energy, and electronic excitation is where an electron in the molecule changes energy levels.
The energy associated with the three types of excitations is very different. Rotational transitions tend to occur at microwave frequencies, vibrational in the infra-red and electronic in the visible to ultraviolet. Of these three only the electronic transitions can be usefully employed in an eye for a variety of reasons. For example rotational and vibration excitations tend to get blurred out in solids and liquids due to interactions between molecules. Also microwaves are too large wavelength to give good vision unless the eye is exceeding large (i.e. the size of a radar dish!). Finally infra-red is strongly absorbed by water, and if your eye contains water that's a problem.
So any useful eye is likely to be based upon electronic excitations, and in fact this is just how our eye works because it detects electronic excitations of the rhodopsin molecules. Electronic excitations of simple atoms and molecules lie in the ultra-violet, but UV photons have a lot of energy and are destructive to tissue (that's why you get sunburn) so they aren't much use for vision. By making molecules with conjugated double bonds the energy of electronic excitations can be lowered into the visible range, and that's what eyes do. Vertebrate optical pigments are derived from vitamin A, and this has conjugated double bonds that form a chromophore. However to get the energy lower than 800nm wavelength would require extended and probably unstable molecules, so this sets a lower limit.
So the answer is basically that electronic transitions are required for vision, and practical considerations restrict the useful wavelengths to the 400 - 800nm range.
A: In addition to the other answers, the visual spectrum of light corresponds to a notch in the atmosphere's absorption spectrum.

Image source: http://en.wikipedia.org/wiki/File:Atmospheric_electromagnetic_opacity.svg
A: In my view (haha) of the world, it is because animals exist at a sort of comfortable medium size with respect to the scale of the things they observe. 
Taking humans as an example, were they to be ten orders of magnitude smaller, light probably wouldn't matter as much. Incidentally, if we consider how extremely small organisms perceive the world, we find that vision is a rather small part of the picture for them. Many of the things they care about fall below the observable threshold of the "visible light" spectrum anyway, as most things lack color at this scale (in particular, most things of genuine concern are smaller than 800nm). The 800nm issue is really one of instrumentation, but there is another issue: the number of synapse than can fit into the brain of such an organism. This is a major problem and forces mammals, reptiles and other beings which we consider to be fully sentient to exist above some minimum size, and it happens that beings of this scale are the ones we think about when we contemplate "observers".
If humans were ten orders of magnitude larger, on the other hand, light becomes too slow for them to be concerned with in terms of observation. Sure, the speed of light might impose a hard limit on the mechanics of thought within a physical mind (at the very least because that is the limit to the speed of electrochemical interactions), but it is hard to conceive of a very large sentient being that concerns itself much with light spectrum observations beyond very primitive tests. Consider the absurdly clumsy experiments in which humans currently engage in what they (arbitrarily) perceive to be the "high energy" range.
Sentient animals happen to occur at a particular scale that makes light-based observations quick, easy, prolific and therefore prevalent in their thinking. In this universe, anyway. That doesn't make it special, that makes it an observation bias.
