# Basics of camera calibration

I tried to find some useful explanation about camera calibration and stereo camera setup in the web but was completely overwhelmed by the hairiness of the content: but seriously though, does it have to be that hairy?

As shown in figure 1 I have a setup consisting of a beam-splitter (BS) and 2 cameras (shown as dark gray planes). The cameras are without objectives and the imaging sensors (5 mm $$\times$$ 5 mm) are directly exposed to the beam.

Consider an arbitrary laser beam i - a ray vector - from source S that propagates along $$z-$$direction. The source S is mounted on an actuator which can rotate only along the $$y$$ axis in $$\pm$$d$$\theta$$ increments. For any angle $$\theta$$, the ray i enters the BS and gets splitted at point X then exits the BS to form images $$I_1$$ and $$I_2$$ in Camera 1 and 2 respectively. I want to determine $$\theta$$ by evaluating $$I_1$$ and $$I_2$$.

Note that camera 1 and 2 are NOT at same distances from the respective BS's exit surface. IF at same distance (see Fig 1, Top view) $$I_1 = \tilde{I}_1$$ and the 2 images ($$I_1$$ and $$I_2$$) would be theoretically indistinguishable. Then if $$\tilde{I_1}$$ is the virtual Image (light gray plane) - replica - of $$I_2$$, the angle of the incident ray i can be calculated using

$$\tan(\theta) = \dfrac{{I}_{1,x} - \tilde{I}_{1,x}}{d}$$

$$\qquad$$ where $$\theta$$ is the angle between i and the positive $$z$$ axis. $$d = |O_1\tilde{O}_1|$$ is the distance between the origins of the 2 cameras.

The BS and the 2 cameras is in a blackbox and the only thing I can vary in this blackbox is is the distance of camera 1 and 2 to the BS which is $$d$$ in the equation above. Also distance between $$S$$ and the blackbox can be varied if necessary.

What are the key points to consider while evaluating the images $$I_1$$ and $$I_2$$ from this setup? How would one calibrate the cameras for this setup? Would pin-eye model of the camera still be valid for this type of setup for camera without lenses?

Any comments will be kindly appreciated.

• "but seriously though, does it have to be that hairy?" As an experimentor by training I chuckled at this. Calibrating real-world systems is complicated by all the things that we wave away when setting up problems at the board. In all but the simplest cases you spend more time, skull sweat, energy, and column-length in your paper dealing with calibration than you do with your basic measurement. And you are not talking about a simple system. Commented Oct 7, 2019 at 15:19

Assuming the two lenses are identical, each lens has a center through which its axis goes, and the axis direction has two degrees of freedom. Another degree of freedom corresponds to rotation around the axis. Moreover, the center of the lens has three degrees of freedom: up/down, left/right, and along the lens axis. It is necessary to get all of those degrees of freedom to coincide in order to achieve perfect alignment- and of course the beamsplitter must be in the right location and be oriented correctly, for a total of 18 degrees of freedom.

If there are no lenses, of course there is no image formed. However, the same degrees of freedom exist: the camera sensors each have six degrees of freedom.

If you are using a lensless system, you would not form

Some of those degrees of freedom are coupled. For example the beamsplitter surface is a plane, and the beamsplitter can slide a little bit along that plane without affecting anything important. Some degrees of freedom are redundant: if one lens is fixed, only the other lens and the beamsplitter need to be adjusted. However, that still leaves ten degrees of freedom to adjust correctly.

Calibration is different from perfect alignment. If you just slap the system together without perfect alignment, it can be calibrated without any further alignment. In that case, it is necessary to map out enough locations of image points in the two camera images to solve the equations relating the relative positions of image points to the misalignment. Because there are at least ten unknowns in the equations, and the equations are nonlinear, you will need to map image points for at least ten source points. It will be a messy math problem. After you've solved for the misalignment values and rearranged the equations, you will be able to solve for source point location as a function of image point locations.

• which lens are you talking about? There is no single lens used in the setup... Commented Oct 7, 2019 at 14:16
• True. But you referred to "cameras" and "images", which suggests that you would add lenses. However, I have edited my answer accordingly. Commented Oct 7, 2019 at 15:44
• what do you mean by: If there are no lenses, of course there is no image formed. ? The laser spot diameter is roughly 1 mm and I dont have lenses to get the images. Its just the cmos senor (5 $\times$ 5 mm) directly exposed to the beam. Commented Oct 7, 2019 at 16:27
• It's not usual practice to use the word "image" to refer to a laser spot, but it does help to know that that's what you mean by "image". Nonetheless, the answer stands. You won't be able to accurately interpret a stereo pair without accounting for all those degrees of freedom in aligning the two "imaging" channels. Commented Oct 7, 2019 at 22:37