3D glasses: how do we tell whether a doubly-rendered image is closer or farther than the screen?

Question

This is a followup to this question: 3D glasses giving the opposite effect to that expected

The current top answer explains that objects perceived as beyond the plane of the screen, as well as object perceived as closer to the viewer than the screen, are displayed by rendering two 2d images slightly apart from each other on the screen. This diagram is used in the explanation.

My question is: how do our brains decide whether an object is in front of or behind the screen, given two images that are apart from each other? How does it know the green dot is far away and the blue dot is close?

Answer 1

As you can see from the sketch you posted, there is exactly one point where the corresponding rays traveling to each eye cross (marked by the solid dots). Our brain sees this light as if it emerged out of this single point (while in fact it came from two spots on the screen).

The brain processes the images provided by both eyes into a perception of three dimensions. This happens because both our eyes see the same images as if we really were looking at a physical three dimensional scene.

Answer 2

I realized what was going on partway through writing the question, but thought I would post it anyway for anyone else who comes along.

If you look at the screen at the movie theatre without your 3D glasses, and see both renderings of an object with both eyes, you won't perceive the object to be farther or closer than the screen: you'll just see two objects or a trippy double-object. It's only when you put on the glasses that you perceive the object to be farther away or closer, because the glasses block half of the rendered images and only show one rendering of the object to each eye.

Now here's where it gets interesting: if the image the right eye sees is the one that is on the right on the screen, and the left eye sees the left image, then the image will appear to be behind of the plane of the screen, or far away. But if the image the right eye sees is the left image, and the left eye is shown the rightmost image (the lines from eye to image cross each other in front of the screen, like with the blue dots in the above diagram), then the image will appear to be close (in front of the plane).

The perceived location of the object is exactly at the point where the two lines intersect, the lines being the ones connecting each eye to its respective image.