Projection matrix in the sensor tilted model in calib3d intrinsic camera model. What for?

1

Hi.
I’m trying to get a deep understanding of OpenCV intrinsic camera model and was reading the OpenCV docs(OpenCV: Camera Calibration and 3D Reconstruction) and the source codes. And I came into a question of why the projection matrix in the sensor tilted model, which is given by

[[R33, 0, -R13],
[0, R33, -R23],
[0, 0, 1]]

, exists. Here R13-R33 are the column vector of the rotation matrix in tilted sensor model.

I understand the rotation matrix which accounts for the sensor tilt. But I don’t understand the meaning of this projection matrix because the “actual” projection by the camera matrix follows this “projection”.

I read the following reference paper written in the OpenCV document but the paper doesn’t give the detailed description of the projection matrix.

H. Louhichi, T. Fournel, J. M. Lavest, and H. Ben Aissia. Self-calibration of scheimpflug cameras: an easy protocol. Meas. Sci. Technol. , 18(8):2616–2622, 2007.

Does anyone know where this projection equation came from? And why you need this?

Thanks.

2

I think I resolved it by myself.
If we regard this as a “projection matrix” we may wonder why you need to apply projection twice (1st with this matrix and 2nd with camera matrix). However, if we regard this as a correction factor to make the parameters consistent, it makes sense.

Let me give one quick example. If the sensor is tilted around y-axis, the corresponding rotation matrix becomes

R = [[cos(th), 0, sin(th)],
[0, 1, 0 ],
[-sin(th), 0, cos(th)]].

And, according to the OpenCV document, the “projection matrix” becomes

P = [[cos(th), 0, -sin(th)],
[0, cos(th), 0 ],
[0, 0, 1 ]]

Now if the camera matrix K is the following,

K = [[f, 0, cu],
[0, f, cv],
[0, 0, 1]]

and if we multiply K by P, we get

K P = [[f cos(th), 0, cu - f sin(th)],
[0, f cos(th), cv ],
[0, 0, 1 ]].

And if we treat this matrix as the camera matrix instead of the original camera matrix, K,
we get a new focal length f cos(th) and new offset vector (cu - f sin(th), cv).

Here both f and f cos(th) are the length between the lens center and the sensor surface but f is the length along optical axis where f cos(th) is the length along the vector normal to the sensor surface. That means if you want to parameterize focal length as the length along optical axis you can use this model where if you want to regard focal length as the length along the normal vector you can safely omit this “projection matrix”. It’s a matter of how you want to define the parameter. And the similar argument can be made for cu and cv.