# Is it possible to identify the point directly in front of an observer in an image?

Consider a pinhole camera with a flat, centered, film pointed directly at the center of the face of a cube. Only the front face of the cube is visible, and each side of the cube in the projected image has the same length.

Suppose the camera is on a track which is perpendicular to the line formed between the camera and the cube.

As the camera is slid to the left several things happen:

• the projection moves to the left
• a new face comes into view
• of the edges making up the front face, the one furthest away appears smallest while the closest edge appears longest

If instead, the camera was rotated:

• part of the film would be closer to the cube and part would be further
• the projected edge on the far part of the film would appear largest.
• no other face of the cube would become visible
• the cube's position on the film would not change

In short, as we rotate and translate the camera, the image changes in different ways.

Once we take a photograph, we can crop the photo. Cropping doesn't affect what the cube looks like, but it can affect where on the film, (with respect to the film's center) the cube lies. We still have several cues about the relationship between the camera and the object but we lost one cue. Is it still possible to identify which point was directly in front of the camera?

• Not if the camera has movements, anyway (but this is probably cheating), and I suspect not in general but I'm not sure. – tfb Jan 24 '17 at 16:59
• I don't think it's possible geometrically, since a pinhole produces no distortion and each part of the image is a faithful reproduction of the object. However, for a uniformly-lit flat object I think there would still be vignetting, both because of intensity falling of with distance and because of angle-dependence of a (digital) sensor, so the point directly in front of the pinhole would be the brightest part of the image even if the image were cropped. -- pwf – pwf Nov 9 '17 at 17:42

If we have

1. 3d knowledge of a scene where at least 1 object which obstructs the view of something behind it,
2. a (possibly cropped) photo taken with a pinhole camera with a flat film.

Then we can determine the original center of the photo.

Call the location of the pinhole, at the time the photo was taken, point P. For any point of the scene {s in scene} to appear in the photo, a line between P and s must not intersect any other object in the scene. An object in the scene, consisting of the set of points {o in object} will block a point s if s, o, P are in a line.

Any movement of P will change the set of blocked and visible points by virtue of changing the set of s, o, P tuples that are colinear.

So, given a photo of a scene {s in scene | s visible}, we know exactly where P is.

A pinhole camera consists of a pinhole P and the film F. The film F can be rotated about P, while keeping P fixed. Can we determine the orientation of F?

Yes. Since the film F is planar, the line perpendicular to F through P will intersect F at FC (the center of the film). Points further from FC will be further from P. Parts of the image projected further from FC, will have been projected further from P and therefore will be scaled more than parts of the image closer to FC. Therefore the relative scaling of parts of the scene is determined by the orientation of F about P.

So a pinhole photo, by virtue of of the visible points, tells us the location of P, and by virtue of the relative scaling of parts of the scene, tells us the orientation F. A cropped photo will provide the same information.

Therefore, we can determine if a photo has been cropped, not preserving center.

This is all a bit handwavy, and I'd love to see a more rigorous proof. If you provide a good one, based on this, I will hand over the bounty.