Steve,
I appear to be stuck. In the code below Debug works but with different results on each run through. With Release mode it will always get hung up and stuck while trying to run calibrateCameraCharuco. Here is the code with just about all for the sample cpp from OCV 4.5.3
Can you spot anything that jumps out at you other than the fact that I am only using one iteration of the ChAruco board?
In creating the board I use DICT_6X6_250 only because I saw this once in an example and assumed that maybe the more markers the better but I have found absolutely nowhere where it is discussed why one dictionary should be selected over another.
// Public to be available in calibrateCameraChAruco
cv::Ptr< cv::aruco::CharucoBoard > pCharucoBoard;
cv::Ptr< cv::aruco::Dictionary > dict;
Mat cameraMatric = Mat(3, 3, CV_32FC1);
Mat distCoeffs;
////////////////////////////
// Create the board based on 2nd screen size
cv::Mat mChAruco(rSecondary.Width(), rSecondary.Height(), CV_8UC1);
dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
int iAcross, iDown = 0;
iAcross = rSecondary.Width() / 100;
iDown = rSecondary.Height() / 100;
pCharucoBoard = cv::aruco::CharucoBoard::create(iAcross, iDown, 0.08, 0.04, dict); //
cv::imwrite("board.png", mChAruco);
// Get this ChAruco board onto the GameBoard window
mToScreenChessBoard = imread("board.png");
////// seperate function CalibrateCameraChAruco() /////////////
// collect data from each frame -- from sample
// ...OpenCV_453\opencv\sources\opencv_contrib-4.x\modules\aruco\samples
std::vector< std::vector< std::vector< Point2f > > > vvvp2AllCorners;
std::vector< std::vector< int > > vviAllIds;
std::vector< Mat > vmAllImgs;
cv::Size imgSize;
// Initialize the detector parameters using default values -- From Sample code
cv::Ptr<cv::aruco::DetectorParameters> detectorParams = cv::aruco::DetectorParameters::create();
int iTotalFrameGrabs = 1;
int iFrameGrabs = 0;
// run 5 times on same image as a test
// run 1 time as a test -- No difference either way
while (iFrameGrabs < iTotalFrameGrabs){
std::vector< int > viIds;
std::vector< std::vector< Point2f > > vvp2Corners, vvp2Rejected;
// detect markers -- From Sample code
aruco::detectMarkers(mCameraImage, dict, vvp2Corners, viIds, detectorParams, vvp2Rejected);
// interpolate charuco vvp2Corners -- from Sample code
Mat mCurrentCharucoCorners, mCurrentCharucoIds;
if (viIds.size() > 0)
aruco::interpolateCornersCharuco(vvp2Corners, viIds, mCameraImage, pCharucoBoard, mCurrentCharucoCorners, mCurrentCharucoIds);
if (viIds.size() > 0) {
vvvp2AllCorners.push_back(vvp2Corners);
vviAllIds.push_back(viIds); // Sample V viIds goes into VV allIds
vmAllImgs.push_back(mCameraImage);
imgSize = mCameraImage.size();
}
iFrameGrabs++;
}
if (vviAllIds.size() < 1) {
Beep(555, 555);
//return false;
}
std::vector<Mat> rvecs;
std::vector<Mat> tvecs;
int iCalibrationFlags = 0;
// iCalibrationFlags = CALIB_FIX_ASPECT_RATIO | CALIB_USE_INTRINSIC_GUESS;
// prepare data for charuco calibration -- from Sample code
int nFrames = (int)vvvp2AllCorners.size();
std::vector< Mat > vmAllCharucoCorners;
std::vector< Mat > vmAllCharucoIds;
std::vector< Mat > vmFilteredImages;
vmAllCharucoCorners.reserve(nFrames);
vmAllCharucoIds.reserve(nFrames);
for (int i = 0; i < nFrames; i++) {
// interpolate using camera parameters -- from Sample code
Mat mCurrentCharucoCorners, mCurrentCharucoIds;
aruco::interpolateCornersCharuco(vvvp2AllCorners[i], vviAllIds[i], vmAllImgs[i], pCharucoBoard,
mCurrentCharucoCorners, mCurrentCharucoIds, cameraMatric,
distCoeffs);
vmAllCharucoCorners.push_back(mCurrentCharucoCorners);
vmAllCharucoIds.push_back(mCurrentCharucoIds);
vmFilteredImages.push_back(vmAllImgs[i]);
}
// This calibrateCameraCharuco with no flags and no TermCriteria works in Debug mode
// with repError of around 0.09xxx and an obviously squared off undistorted image
// HOWEVER, in Release mode it will hang along with the attempts below.
double repError =
aruco::calibrateCameraCharuco(vmAllCharucoCorners, vmAllCharucoIds, pCharucoBoard, imgSize,
cameraMatric, distCoeffs, rvecs, tvecs);
cv::Mat mImageUndistorted;
undistort(mCameraImage, mImageUndistorted, cameraMatric, distCoeffs);
imshow("winUndistorted", mImageUndistorted); // Unsidtorted cameraWindow
waitKey(1);
// The above fails in Release mode as well as these attempts
// double repError =
// aruco::calibrateCameraCharuco(allCharucoCorners, allCharucoIds, pCharucoBoard, imgSize,
// cameraMatric, distCoeffs, rvecs, tvecs, iCalibrationFlags, TermCriteria(TermCriteria::EPS | TermCriteria::COUNT, 10, 1));
// I have tried with iCalibrationFlags as 0, CALIB_FIX_ASPECT_RATIO, with and without | CALIB_USE_INTRINSIC_GUESS
// and Release mode will just hang while running the calibrateCameraCharuco
// In both cases, Debug and Release, the distCoeff is 0,0,0 NULL, NULL, NULL,...
// doesn't matter in Debug but I have a feeling that it matters in Release
// Should this have been filled in in interpolateCornersCharuco above?
// In debug I hit a repError of 4.5 and in that case the distCoeffs was filled in
// but, of course, the undistorted board was still fish eyed.
// next time in debug the error was .5 and the undistorted board was still fisheyed.
//
PS. I also tried the rational model for the flag but that did not help either.
PPS. I just realized that I had not transferred the cameraMatric (sp) from my previous calibrateCamera to this calibrateCameraCharuco. I added this before the call to interpolateCornersCharuco above
cameraMatric.ptr<float>(0)[0] = mCameraImage.cols / 2;
cameraMatric.ptr<float>(1)[1] = mCameraImage.cols / 2;
cameraMatric.ptr<float>(0)[2] = mCameraImage.cols / 2; // Cx = Half the width
cameraMatric.ptr<float>(1)[2] = mCameraImage.rows / 2; // Cy = Half the height
cameraMatric.ptr<float>(2)[2] = 1.0; // ?
That seems to have fixed all of my problemsā¦Jeeeeze, it is always something.
Ed
Ed
