# Problem with Charuco Calibration

**URL:** <https://forum.opencv.org/t/problem-with-charuco-calibration/18391>\
**Category:** C++\
**Tags:** calib3d, aruco\
**Created:** [July 31, 2024, 9:06pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391 "2024-07-31T21:06:12Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Buddy0193](https://avatars.discourse-cdn.com/v4/letter/b/f9ae1b/32.png) [@Buddy0193](https://forum.opencv.org/u/Buddy0193)\
**Post date:** [July 31, 2024, 9:06pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/1 "2024-07-31T21:06:12Z")

</div>

Hey everyone! I’m currently trying to calibrate my camera with 7 given images (see [here](https://drive.google.com/drive/folders/1ZDpnl3i5cgbes1EdB1FBXYqqHeH_HRsJ?usp=sharing)) of a Charuco-Board [following this example from the documentation](https://docs.opencv.org/4.x/da/d13/tutorial_aruco_calibration.html). However, the result is so bad, that I think I got something fundamentally wrong here.

 ![result](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/f/f12df0de5165b9a9a4db41f7c64ca6ebe6f9a039.jpeg)

Can someone spot my mistake?

```cpp
#include <iostream>
#include <vector>
#include <opencv2/calib3d.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/objdetect/charuco_detector.hpp>

using namespace std;
using namespace cv;

int main() {
    // Define board parameters
    int squaresX = 11;
    int squaresY = 8;
    float squareLength = 15.0f; // in mm
    float markerLength = 11.0f; // in mm

    // Create CharucoBoard and detector
    aruco::Dictionary dictionary = aruco::getPredefinedDictionary(aruco::DICT_4X4_50);
    aruco::CharucoBoard board(Size(squaresX, squaresY), squareLength, markerLength, dictionary);
    aruco::CharucoParameters charucoParams;
    aruco::DetectorParameters detectorParams;
    aruco::CharucoDetector detector(board, charucoParams, detectorParams);

    // Vectors to store the detected corner positions and ids
    vector<Mat> allCharucoCorners, allCharucoIds;
    vector<vector<Point2f>> allImagePoints;
    vector<vector<Point3f>> allObjectPoints;
    Size imageSize;

    int successfulImages = 0;

    // Process all calibration images
    for (int i = 1; i <= 7; i++) {
        string filename = "assets/iPhone/cal_0" + to_string(i) + ".png";
        Mat image = imread(filename);
        if (image.empty()) {
            cerr << "Could not read image: " << filename << endl;
            continue;
        }

        successfulImages++;
        imageSize = image.size();

        vector<int> markerIds;
        vector<vector<Point2f>> markerCorners;
        Mat currentCharucoCorners, currentCharucoIds;

        // Detect ChArUco board
        detector.detectBoard(image, currentCharucoCorners, currentCharucoIds);

        if (currentCharucoCorners.total() > 3) {
            vector<Point3f> currentObjectPoints;
            vector<Point2f> currentImagePoints;
            board.matchImagePoints(currentCharucoCorners, currentCharucoIds, currentObjectPoints, currentImagePoints);

            // For debugging
            Mat imageWithCorners = image.clone();
            aruco::drawDetectedCornersCharuco(imageWithCorners, currentCharucoCorners, currentCharucoIds);
            imshow("Detected Corners - Image " + to_string(i), imageWithCorners);
            waitKey(500); // Display each image for 500ms
            // End debugging

            if (!currentImagePoints.empty() && !currentObjectPoints.empty()) {
                allCharucoCorners.push_back(currentCharucoCorners);
                allCharucoIds.push_back(currentCharucoIds);
                allImagePoints.push_back(currentImagePoints);
                allObjectPoints.push_back(currentObjectPoints);
            }
        }
    }

    cout << "Successfully processed " << successfulImages << " out of 6 images." << endl;

    if (allCharucoCorners.size() < 4) {
        cerr << "Not enough corners for calibration. Need at least 4 valid images." << endl;
        return 0;
    }

    // Calibrate camera
    Mat cameraMatrix, distCoeffs;
    vector<Mat> rvecs, tvecs;
    int calibrationFlags = 0;
    double repError = calibrateCamera(allObjectPoints, allImagePoints, imageSize,
                                      cameraMatrix, distCoeffs, rvecs, tvecs, calibrationFlags);

    // Print calibration results
    cout << "Reprojection error: " << repError << endl;
    cout << "Camera matrix: " << endl << cameraMatrix << endl;
    cout << "Distortion coefficients: " << endl << distCoeffs << endl;

    // Undistort and display an image
    string filename = "assets/iPhone/cal_01.png"; // Using the first image
    Mat image = imread(filename);
    if (image.empty()) {
        cerr << "Could not read image for undistortion: " << filename << endl;
        return -1;
    }

    Mat undistorted;
    undistort(image, undistorted, cameraMatrix, distCoeffs);

    // Display original and undistorted images
    namedWindow("Original", WINDOW_NORMAL);
    namedWindow("Undistorted", WINDOW_NORMAL);
    imshow("Original", image);
    imshow("Undistorted", undistorted);

    cout << "Press any key to close the image windows..." << endl;
    waitKey(0);

    return 0;
}

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [July 31, 2024, 11:40pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/2 "2024-07-31T23:40:28Z")

</div>

you might want to show at least thumbnails of your seven calibration pictures

---

<div class="post-metadata">

**Author:** ![Steve\_in\_Denver](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Steve\_in\_Denver](https://forum.opencv.org/u/Steve_in_Denver)\
**Post date:** [August 1, 2024, 4:48pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/3 "2024-08-01T16:48:02Z")

</div>

What error value did you get from the calibration? I suspect it was pretty high? (I’d say shoot for a value below 1.0 - if you are getting huge error values it tells you that the calibration didn’t model the data very well, which could lead to results like you are getting.)

The obvious issues I see with your calibration process:

1. calibration target isn’t very flat. print it again and adhere it to a piece of glass or something else really flat. or at a minimum re-print it and avoid the wrinkles.
2. you need to move the camera (or the target) in a way that gets more perspective distortion in at least some of the images. Most of your images look to be “straight on” shots - try some with 30 degree rotation between the calibration target and the image plane.
3. you need more data from close to the corners of the image. Since you are using the Charuco target you don’t need to see the whole thing - position the camera / target so you get points as far into the corners as you can. This will greatly help with getting accurate distortion parameters.

---

<div class="post-metadata">

**Author:** ![Buddy0193](https://avatars.discourse-cdn.com/v4/letter/b/f9ae1b/32.png) [@Buddy0193](https://forum.opencv.org/u/Buddy0193)\
**Post date:** [August 2, 2024, 2:58pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/4 "2024-08-02T14:58:30Z")

</div>

Yes, I get crazy high reprojection errors, for the example above it was 57.7803.

I tried again with your advices - reprojection error of 271.532. By now, I’m pretty certain the script contains an error.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [August 2, 2024, 5:42pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/5 "2024-08-02T17:42:26Z")

</div>

you might want to show at least thumbnails of your seven calibration pictures

---

<div class="post-metadata">

**Author:** ![Steve\_in\_Denver](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Steve\_in\_Denver](https://forum.opencv.org/u/Steve_in_Denver)\
**Post date:** [August 2, 2024, 7:19pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/6 "2024-08-02T19:19:41Z")

</div>

Make sure the board you define in the script is the same as the one you are using. For example, it’s fairly common to interchange the X/Y size of the board and then get nonsensical results. a 5x7 physical board and a 7x5 board in the code.

Also I always draw the detected points and back-projected / estimated point locations to the original images. This way you can get a better visual understanding of what is going wrong. Super helpful, highly recommended.

---

<div class="post-metadata">

**Author:** ![Buddy0193](https://avatars.discourse-cdn.com/v4/letter/b/f9ae1b/32.png) [@Buddy0193](https://forum.opencv.org/u/Buddy0193)\
**Post date:** [August 9, 2024, 10:48pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/7 "2024-08-09T22:48:16Z")

</div>

Thanks for your tips! I ended up trying with the board and values described on [GitHub](https://github.com/opencv/opencv_contrib/tree/db16caf6ceee76b43b94c846be276e92a43e9700/modules/aruco/tutorials/aruco_calibration/images), just to see if my implementation was correct. It was.

I tried reproducing the images shown in the repository with my own board, double checking if the parameters were correct - it still didn’t work.

I then switched over to a ChArUco board using DICT\_6X6\_250 instead of DICT\_4X4\_50. Suprisingly, this made the calibration a lot better. I’m not sure why this makes a difference, but…it works.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [August 9, 2024, 10:54pm UTC](https://forum.opencv.org/t/problem-with-charuco-calibration/18391/8 "2024-08-09T22:54:26Z")

</div>

that’s what I thought. image corners aren’t covered by points of the pattern.