Camera calibration RMS error increases with more calibration frames

C++
1

I’m trying to calibrate a camera using OpenCV in C++. The code that I’ve written works as far as capturing the images of the chessboard and running the calibration. Only the results that I get don’t make sense. I can never seem to get the RMS error to go below 50 at 720p, and the error usually increases with the more frames I use. I’ve posted my calibration code below, and any advice would be greatly appreciated.

std::vector<std::vector<cv::Point3f>> objectPoints;
    std::vector<cv::Point3f> points;
    for (int i = 0; i < mCheckerboardSize.width; i++)
        for (int j = 0; j < mCheckerboardSize.height; j++)
            points.push_back(cv::Point3f{(float)i, (float)j, 0});

    std::vector<std::vector<cv::Point2f>> imagePoints;

    cv::Mat cameraMatrix, distCoeffs, rotations, translations;

    for (int i = 1; true; i++)
    {
        std::cout << "Press ENTER to capture frame " << i << " (enter any character first to end calibration)";
        if (std::cin.get() != '\n')
            break;

        while (mFrameLock.exchange(true, std::memory_order_acquire))
            ;
        cv::Mat frame = mFrame.clone();
        mFrameLock.store(false, std::memory_order_release);

        mSize.width = frame.cols;
        mSize.height = frame.rows;

        cv::cvtColor(frame, frame, cv::COLOR_BGR2GRAY);

        std::vector<cv::Point2f> corners;
        bool result = cv::findChessboardCorners(frame, mCheckerboardSize, corners, 0);
        if (!result)
        {
            i--;
            std::cerr << "Fail\n";
            continue;
        }

        objectPoints.push_back(points);
        imagePoints.push_back(corners);

        double error = cv::calibrateCamera(objectPoints, imagePoints, mSize, cameraMatrix, distCoeffs, rotations, translations, cv::CALIB_FIX_ASPECT_RATIO | cv::CALIB_ZERO_TANGENT_DIST);

        std::cout << "Ok (error = " << error << ")\n";
    }
    mRunning = false;
    thread.join();
    mCamera.release();

    if (imagePoints.size() < 1)
    {
        std::cerr << "No calibration frames\n";
        return -1;
    }

    cv::Mat optimalCameraMatrix = cv::getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, mSize, 1, mSize);

    std::stringstream filename;
    filename << "./camera" << mCamera.id() << ".yml";
    cv::FileStorage file{filename.str(), cv::FileStorage::WRITE};
    file << "cameraMatrix" << optimalCameraMatrix;
    file << "distCoeffs" << distCoeffs;
    file.release();
2

show us some of those images.

always store those pictures you capture at runtime, so you can analyze the dataset and run calibration again, on the same data.

the code is irrelevant, unless you have reason to suspect that the code is at fault.