It's a bit more biology than physics. I understand the tehnology much the same way as you do (although not all 3D techniques work with polarization, but that's irrelevant to the explanation). The idea of 3D glasses is to make your eyes see two different images and thus fool your brain into thinking those different images come from the spatial perception of an object. That is, your two eyes deliver slightly displaced perspectives on the same object. Effectively, there is a projection of the 3D object on the two flat (i.e. 2D) surfaces in your retinas. Your brain considers all this to be the result of parallax and recovers a 3D image like everything that hits your eyes in everyday life.
This works fine as long as you are far enough away from the screen so that your brain can still make sense of the two images. But once you get very close, your brain receives images it can no longer make sense of as coming from two slightly displaced perspectives on the same object, but rather that your two eyes are seeing two different objects.
Not sure about this answer, happy for improvement suggestions. Also, I need to check this out next time I am in a 3D theater (which could be 2022 at the rate things are moving).