The highly interesting thing is, brown absorbs all other colors but reflects red and green.
This may be a source of your misunderstanding.
The way humans perceive color is by having three types of photoreceptors in their eyes. Each of them has a broadish response function which means, that the "green" receptors react well to pure green light (as in 555nm), but also are somewhat excited by yellow (589nm) and hardly respond to red (>600nm). The same reasoning goes to the "red" receptors, which respond well to red (632nm, for example), but the yellow (589nm) also affects them somewhat.
Therefore, when you shine a red and green light onto a white screen, it reflects both red and green photons, exciting both "red" and "green" photoreceptors in your eye, which the brain interprets then as yellow. But similarly, you can simply illuminate the screen with a yellow light and get a 589nm wave, to which both mentioned types of receptors are sensitive. As a result, they're both excited and the brain again interprets (this time correctly) this as yellow. (see first figure)
Now the way pigments mix is slightly different. A pure pigment absorbs all light, except light of its color. Again, not perfectly, but with a relatively broad response, i.e. a green pigment may reflect some blues and yellows, but the main part is centered on the green. Similarly, a red pigment absorbs all, but red. So when you mix the pigments, qualitatively you're summing up their absorption functions. The presence of red prevents you from seeing much of the reflected green light, conversely, the presence of green prevents you from seeing much of the reflected red. However, in between, where the yellow lies, their response functions overlap and they reflect (although strongly suppressed), some yellow.
This dark yellow, smeared over a wide portion of the spectrum is perceived as brown. (see second figure)
Brain identifies "red" and "green" receptors to both be strongly excited in roughly equal parts. It figures, the wavelength of incident light to lie at the point where they are equally sensitive and concludes, that incident light is yellow. However, it may just as well be equal amount of red and green photons.
Red and green pigments have - qualitatively - these kind of reflection spectra. When you mix them together, you get the worst of both worlds: neither green, nor red is reflected particularly well, some residual reflection remains in between.