First, to clarify: I'm not asking if perhaps there's a moon that we haven't found yet. The question is, theoretically, would the earth be able to have another stable moon in addition to the current one? Or, if the orbit couldn't be stable, why not? How large/small of a moon would it be able to have? And how do we know all this?
Jupiter has over 60 moons, and the dozens of man-made satellites (along with the thousands of pieces of space-trash) orbiting the Earth could be considered tiny moons.
Earth's main Moon would disrupt the orbit of anything smaller at certain radii. The particular disrupted orbits are called resonances, and occur where the orbital period in question divided by the period of the Moon reduces to a small fraction, like 1/2, 2/3, etc. It comes from the Moon giving a predictable tug on objects at those resonance orbits, similar to pushing a child on a swing higher and higher so that you eventually pitch him right out onto the ground. (Do not try this at home.)
The principle is spectacularly illustrated by Saturn's rings. (Some of*) the dozens of dark tracks indicate where one of the moons has a resonance. http://upload.wikimedia.org/wikipedia/commons/thumb/b/b1/Saturn%27s_rings_dark_side_mosaic.jpg/2200px-Saturn%27s_rings_dark_side_mosaic.jpg
[EDIT: To answer more of your questions, there are lots of stable orbits, and the Moon can also act to stabilize, not just destabilize orbits- see http://en.wikipedia.org/wiki/Trojan_(astronomy) . Any stable orbit would have to be above the top of the atmosphere, about 100 km, or else friction with atmospheric gas would drag it down to Earth. Moons could range from specks of dust up to potentially extremely large, even as large as the first Moon, but at great size the options for stability become much more limited. See http://en.wikipedia.org/wiki/Euler%27s_three-body_problem. All of this arises from classical, Newtonian mechanics. To greatly simplify history, Tycho Brahe took a lot of data on planet positions in the sky, Kepler took those observations and developed his purely empirical laws of orbital motion, and Newton took those empirical laws and developed a theoretical framework to account for them.]
*Some gaps are due to other or unknown factors, but for the most part...
Some studies have suggested that the Earth may have had a second tiny moon, which later crashed into and merged with the bigger current Moon. This might explain the lopsided back and front faces of the Moon. The second moon may have orbited the Earth for between 10 million and 100 million years. This little moon was likely about 750 miles wide, which would've made it larger than the dwarf planet Ceres in the Asteroid Belt.
Earth Had 2 Moons That Crashed to Form 1, Study Suggests | Moon Formation, Facts & Figures | How Earth's Moon Formed
Secondly, currently there may be an asteroid in resonant orbit of Earth, called 3753 Cruithne. Now whether this could be considered a Moon or something else is hard to decide. It does orbit around the Sun in exactly one year like the Earth and the Moon do. It follows a strange horseshoe type orbit around the Earth.
How Many Moons Does Earth Have?