Calibration PTZ camera with OpenCV's calibrateCamera. How to fix certain parameters across frames?

Python
, ,
1

I have N images from a PTZ camera with some point correspondences in each frame.

The problem is the images are taken at different zoom.

I am wondering how to use cv2.calibrateCamera for these N images so that some parameters are fixed and some not.

I would like these parameters to NOT be fixed:

  • fx=fy can change from image to image. (zoom with cv2.CALIB_FIX_ASPECT_RATIO for simplicity)
  • rotation vectors to change from image to image

I would like to fix these parameters for all images:

  • optical centers (cx,cy)
  • camera position as translation vectors tvecs
2

zoom causes multiple pieces of “the lens” to move.

you can forget about calibration if zoom changes at all. that immediately makes the problem explode in complexity.

OpenCV’s camera calibration function lets you nail down (fix) a few parameters. the docs will say which those can be.

3

Thanks @crackwitz for your reply. I understand that zoom causes lots of problems for calibration. Currently the approximation go-to method for video with zoom involves accounting for zoom with focal distance but keeping the position of the camera fixed.

I went through OpenCV’s clibratecamera documentation many times and cannot find HOW to fix parameters like tvecs. Could you point me in the right direction?

4

those can’t be fixed in OpenCV camera calibration.

if you are mathematically sure that you must fix those and nothing else will do, then you’ll have to implement the optimization yourself.

5

Ok, thanks @crackwitz. I had another question if you don’t mind.

cv2.caliblrateCamera outputs different rvecs and tvecs for every image in the input array. If the camera pose is the same why would rvecs and tvecs be different for every view?

As I understand they define the position and orientation of the camera so to me it seems like they should be the same for every shot of the camera. Wha am I missing?

6

to the resp. chessboard, in a certain image.

that’s simply wrong.
you might have expected, this is some kind of ‘world’ position – very much NOT SO !

so again, it’s unclear, why you would want to ‘fix’ those (it’s impossible)
also, the rvecs/tvecs obtained from calibration are only useful to check the calibration. (they have NO meaning without checkboards later on …)

1 Like
7

crosspost:

1 Like
8

Thanks @berak , are they not defined with respect to the origin (0,0,0) of the coordinate system of the chessboard? That would essentially indicate the positioin of the camera, as -np.transpose(cv2.Rodrigues(rvecs))@tvecs.

In my case I have a video that was shot from the camera in the same position. But the camera is zooming and rotating. Only it’s position stays the same. I assumed that rvecs and tvecs were rotation and position of the camera and that’s why I wanted to keep tvecs constant and rvecs flexible. …

9

true, but only as long as there IS a chessboard (no more so, later)

to retrieve the camera’s position / orientation (towards something) after the calibration, you’ll have to use solvePNP() or similar, with a known 3d object (like another chessboard)