It's possible to do cv2.stereoCalibrate() with ChArUco-Board?

Hello people,

I am currently working on a stereo calibration of two cameras. I am currently using a normal checkerboard. I also get correct results, for example I measure the distance of my cameras and compare them with the translation matrix.

Now I have heard that it is more robust if I use the ChArUco board. I tried to find something on the internet but there are really almost no tutorials. I then tried to program it myself, however I get nonsense results. Is it even possible to do a stereo calibration with a ChArUco board? I have added the main party of my code, maybe I did something wrong and someone can give me a hint.

Thank you.

pair_images = zip(images_l, images_r)

for images_l, images_r in pair_images:
    # Left Image Points
    img_l = cv2.imread(images_l)
   grayL = cv2.cvtColor(img_l, cv2.COLOR_BGR2GRAY)
   corners_L, ids_L, rejected_L = cv2.aruco.detectMarkers(grayL, aruco_dict, 
                                                          parameters=arucoParams)
   resp_L, charuco_corners_L, charucos_ids_L =  
                                 cv2.aruco.interpolateCornersCharuco(corners_L, ids_L, grayL, board)

   # Right Image Points
   img_r = cv2.imread(images_r)
   grayR = cv2.cvtColor(img_r, cv2.COLOR_BGR2GRAY)
   corners_R, ids_R, rejected_R = cv2.aruco.detectMarkers(grayR, aruco_dict, 
                                                         parameters=arucoParams)
    resp_R, charuco_corners_R, charucos_ids_R = 
    cv2.aruco.interpolateCornersCharuco(corners_R, ids_R, grayR, board)


   objpoints_L, imgpoints_L = cv2.aruco.getBoardObjectAndImagePoints(board, charuco_corners_L, charucos_ids_L)
   objpoints_R, imgpoints_R = cv2.aruco.getBoardObjectAndImagePoints(board, charuco_corners_R, charucos_ids_R)

   if resp_L == resp_R and (resp_L and resp_R) > 1:
    corners_list_L.append(charuco_corners_L)
    corners_list_R.append(charuco_corners_R)

    id_list_L.append(charucos_ids_L)
    id_list_R.append(charucos_ids_R)

    objpoints.append(objpoints_L)
    imgpointsR.append(imgpoints_R)
    imgpointsL.append(imgpoints_L)
    # Draw and display the corners
    cv2.aruco.drawDetectedCornersCharuco(img_l, charuco_corners_L, charucos_ids_L, (255,0,0))
    cv2.aruco.drawDetectedCornersCharuco(img_r, charuco_corners_R, charucos_ids_R, (255,0,0))

    cv2.imshow('imgL', img_l)
    cv2.imshow('imgR', img_r)
    cv2.moveWindow("imgR", 800, 0)
    cv2.waitKey(200)
    else:
       print("Chessboard couldn't detected. Image pair: ", images_l, " and ", images_r)
       continue

 cv2.destroyAllWindows()

ret, M1, d1, M2, d2, R, T, E, F = cv2.stereoCalibrate(objpoints, imgpointsL, imgpointsR, mtxL, 
                       distL, mtxR, distR,(640, 480))
print(T)

Quick answer without looking at the code.

Yes, it is possible. Camera calibration needs input 2D coordinates from whatever methods. The advantage of using ChArUco board vs classical calibration board should be:

  • with classical calibration board, the board must be fully visible in the image
  • more accurate 2D corners coordinates with ChArUco board (I think, to be checked)

Hey Hans,

had the same problem and solved it recently. The problem is coming from the object points given by

whose y-axis are mirrored in case of ChArucos which causes the stereo calibration to fail. You can easily solve the problem by inserting:

newChessboardCorners = board.chessboardCorners
newChessboardCorners[:,1] = (CHARUCOBOARD_COLCOUNT*squareLength) - newChessboardCorners[:,1]
 
objpoints = []
for idx in charucos_ids:
    objpoints.append(newChessboardCorners[idx])
objpoints = np.asarray(objpoints)
 
 imgpoints = charuco_corners

Hello Hasn and mt-cv,

I am also facing the same issue in the later stages of the calibration. I have estimated the camera matrices and the distortion coefficients using the funtion:

aruco.calibrateCameraCharuco

Then I am trying to calculate the object and image points inorder to stereo calibrate, by using:

aruco.getBoardObjectAndImagePoints

I assume, after using the above function, I should be getting object points and image points, but I am getting an unexpected error. Could you please help me to resolve it?

**objpoints_L, imgpoints_L = aruco.getBoardObjectAndImagePoints(board, corners_all, ids_all)**
**cv2.error: OpenCV(4.5.4) :-1: error: (-5:Bad argument) in function 'getBoardObjectAndImagePoints'**
**> Overload resolution failed:**
**>  - detectedIds is not a numpy array, neither a scalar**
**>  - Expected Ptr<cv::UMat> for argument 'detectedIds'**

Hi, I am facing a similar problem but even after inserting your code, i am still getting nonsense results. Can you please share your code. How did you manage to do stereo calibration using a charuco board? Thanks!