The leading answer to this (closed) question gives some good reasons why IR vision did not develop widely across the animal kingdom. To paraphrase, sensing even the near-IR spectrum would require a different type of sensor compared to more or less regular chromophores, and there would be limited evolutionary pay-off for detection. The advantage gained over vision in our own visible range is not necessarily worth it, since everything is awash in IR at life-friendly temperatures. The latter idea is debatable though, since there are species of snakes and beetles that did develop IR sensing, although with organs separate from their eyes (the much loathed bed bugs are also adept at IR sensing o_o). But from an evolutionary point of view, IR sensing is obviously a much later development than regular vision.
It may simply be that protein structures that can serve as good chromophores in the visible and near UV range, and provide useful frequency resolution, are statistically much more available than anything that can function well for the IR spectrum, and in a complex, liquid-based environment at that (think spectral broadening).
[ As an interesting aside idea, this paper explained that we humans are actually capable of seeing near-IR radiation beyond 1000nm under the right conditions - by way of two-photon excitation of rhodopsin. See here for the funny physics story behind this little discovery. ]
As for limitations to the UV range, many species, including butterflies, bees, fish, birds, and even mammals (reindeer) do have near-UV vision (UV-A band), well beyond the 400nm limit for human vision. But biological vision at shorter wavelengths, especially beyond UV-B, seems to be as useless as mid- and far-IR vision, although for different reasons. As far as terrestrial life is concerned, UV radiation is a potent source of mutations and is generally disrupting for biological processes (conformational transitions, radicals). Existing chromophores get destroyed by shorter UV, so UV vision would have to rely on different sensors. On the other hand, the advantage gain would be minimal again, since most current species actually require low UV environments where UV vision may not help much.