Undefined cv::Mat Behavior

Hello all!

I’m having a bit of an issue with the behavior of a couple cv::Mat variables in my program (C++). I am working with camera calibration, and am trying to read my saved camera intrinsics from a file which simply has each cell value saved to its own line in order. When reading the variable from a stream as follows, it seems to work (see code P1).

As expected, the “cam” matrix is populated with the double values read from the stream. Due to changes in my program’s structure, using the previous code is no longer applicable. Implementing a similar piece of code into a class, where the values are saved as class variables, I run into the issue where the matrix does not properly populate (see code P2).

In the code snippet above, I have confirmed that the correct values are loaded into the private class variables (px, u0…), however, an attempt to populate the array directly with these variables leads to an empty matrix.

Something stranger yet: when I do not use the variables (which, once again, I have already confirmed have the right values in them), but use constants instead, it works (see code P3).

EXAMPLE CODE

As things stand, I can use this work-around and simply write in the variables by hand, but that is not a long-term solution. Does anyone have any insight as to what could be causing this (from my perspective) undefined behavior?

Thanks in advance for any replies!
C.Fox

please do not post screenshots of code, but TEXT instead, thank you.

also, we need to see your real code, not an “idea of it” with elipses like //...

then, we don’t know, how your cameraclass looks like, and cameraclass camera = new cameraclass() isn’t valid c++

again, if we can’t try your code, we cannot help.

I don’t see the variable type for px and u0 - could that be the problem (truncation/coercion)?

Bill

Thank you very much for the reply and the advice. I wanted to try and avoid posting huge blocks of code as text, but I understand where you’re coming from! I’ll make sure to use text for future code!

The file containing the camera calibration data, contains only the following lines:

camera.xml file
<!-- Pixel ratio-->
<px>606.20444013030078</px>
<py>612.35364294795841</py>
<!-- Principal point-->
<u0>507.66415593888757</u0>
<v0>521.7748417433304</v0>
<!--Undistorted to distorted distortion parameter-->
<kud>-0.24054321189834804</kud>
<!--Distorted to undistorted distortion parameter-->
<kdu>0.40028260701410684</kdu>

As for the camera class, it is simply written as follows. For sake of brevity, I’ve combined both the *.h and *.cpp files into the header:

CameraClass (h/cpp combined)
#include <opencv2/core.hpp>
#include <fstream>
#include <iostream>
#include <unistd.h>
#include <string>

class CameraClass
{
public:
    CameraClass();
    // core functions
    bool getValues() {
        // create path variable pointing to intrinsic file
        std::string intrinsicPath = getcwd(nullptr, 0);
        intrinsicPath = intrinsicPath.substr(0, intrinsicPath.find_last_of("/")) + "/TestLoadCalib/Calibration/camera.xml";
        std::cout << intrinsicPath << std::endl;

        std::ifstream stream(intrinsicPath);

        if (stream.is_open()) {
            std::string line;
            // parse through file for intrinsic values
            if (int(line.find("<px>")) >= 0) {
                line = line.substr(line.find("<px>") + 4, line.find("</px>") - (line.find("<px>") + 4));
                px = std::stod(line);
            }
            else if (int(line.find("<u0>")) >= 0) {
                line = line.substr(line.find("<u0>") + 4, line.find("</u0>") - (line.find("<u0>") + 4));
                u0 = std::stod(line);
            }
            else if (int(line.find("<py>")) >= 0) {
                line = line.substr(line.find("<py>") + 4, line.find("</py>") - (line.find("<py>") + 4));
                py = std::stod(line);
            }
            else if (int(line.find("<v0>")) >= 0) {
                line = line.substr(line.find("<v0>") + 4, line.find("</v0>") - (line.find("<v0>") + 4));
                v0 = std::stod(line);
            }
            else if (int(line.find("<kud>")) >= 0) {
                line = line.substr(line.find("<kud>") + 5, line.find("</kud>") - (line.find("<kud>") + 5));
                kud = std::stod(line);
            }
        }
        else {
            std::cout << "There was an issue opening the file!" << std::endl;
            return false;
        }
        return true;
    }
    bool fillMatrices() {
        // initialize camera matrix
        cam = cv::Mat::zeros(3, 3, CV_64F);
        //       | px    0  u0 |
        // cam = |  0   py  v0 |
        //       |  0    0   1 |
        cam.at<double>(0, 0) = px;
        cam.at<double>(0, 2) = u0;
        cam.at<double>(1, 1) = py;
        cam.at<double>(1, 2) = v0;
        cam.at<double>(2, 2) = 1.0;

        // initialize distortion matrix
        dist = cv::Mat::zeros(5, 1, CV_64F);
        dist.at<double>(0, 0) = kud;
        return true;
    }
    void printMatrices() {
        std::cout << "Camera Matrix:" << std::endl;
        std::cout << cam.at<double>(0, 0) << " | " << cam.at<double>(0, 1) << " | " << cam.at<double>(0, 2) << std::endl;
        std::cout << cam.at<double>(1, 0) << " | " << cam.at<double>(1, 1) << " | " << cam.at<double>(1, 2) << std::endl;
        std::cout << cam.at<double>(2, 0) << " | " << cam.at<double>(2, 1) << " | " << cam.at<double>(2, 2) << std::endl;

        std::cout << "Distortion Matrix:" << std::endl;
        std::cout << dist.at<double>(0, 0) << " | " << dist.at<double>(0, 1) << " | " << dist.at<double>(0, 2) << " | " << dist.at<double>(0, 3) << " | " << dist.at<double>(0, 4) << std::endl;
    }
    // camera matrices
    cv::Mat cam;
    cv::Mat dist;
private:
    // camera intrinsic values read from file
    double px = 0.0;
    double py = 0.0;
    double u0 = 0.0;
    double v0 = 0.0;
    double kud = 0.0;
};

The main program simply creates an instance of the camera class, calls the function to populate the camera and distortion matrices, and displays these matrices with a print function. (This is for later use with the cv::undistort() function for images taken with the calibrated camera!)

Main Program
#include "cameraclass.h"

int main()
{
    CameraClass *camera = new CameraClass();
    camera->getValues();
    camera->fillMatrices();

    camera->printMatrices();
    delete camera; // clean up
    return 0;
}

Perhaps I should have the title of this question changed to reflect the fact that the problem lies more-so with the further implementation of these matrices, and not the population of the matrices itself. After further deliberation and debugging, I have found that the matrices do in fact contain the values of the variables passed to them. However, the issue still stands, that these variables are not read properly from the matrix when using them with the aforementioned cv::undistort() function.

To reiterate, when hard-coding the variable by hand as such:

dist.at<double>(0, 0) = -0.24054321189834804;

it works as intended. However, for whatever reason, when passing a double variable which contains the exact same number:

double kud = -0.24054321189834804;
dist.at<double>(0, 0) = kud;

it does not have the desired effect. During debugging, I’ve tried to check the value of the variable in the matrix, and even tried using the standard manipio library to set the precision of the output variable when printing it out, and have ensured that the number in its entirety is held within the matrix. Regardless, the matrices are not properly being read and utilized when the variable-version is used. I’m really at my wits end with this one!

Thanks for any insight anyone can provide!

Best regards,
C.Fox

Thank you for your response Bill!

The px and u0 variables were class variables, and already defined in the class as double. I should have been clearer on that!

After further deliberation and bug testing, I have found that the matrix is indeed being populated with variables. The problem is with the further implementation of those very matrices.

My program uses the camera and distortion matrices in order to rectify the image taken by the camera, using the cv::undistort() function. When I try to write the variable directly as such:

distortion_matrix.at<double>(0, 0) = -0.24054321189834804;

the rectification process works as intended. However, when I use a variable with the exact same number assigned to it, as such:

double kud = -0.24054321189834804;
distortion_matrix.at<double>(0, 0) = kud;

I receive a very poorly rectified image.

I think you may be on to something with the possible “truncation” of the double variable. I remember reading somewhere that double values have something like a fifteen decimal digit precision, yet the variable I am defining has nearly twenty.

I’ll have to do some more research, to find out if that’s the case!

Cheers,
C.Fox

Not sure it matters (grasping straws) but make sure the whole development chain is 64 bit.

again, please post a Minimal Reproducable Example (as text, bloody !!), that others can actually try.

don’t waste our time with speculations, please.

I highly doubt your issue is due to anything going on beyond the 15th digit. not even the most advanced physics on earth can pin down natural constants to 15 digits.

my bet is that something trashes your stack and/or heap and that corrupts your data.

if that’s the case, posting little snippets of code will not allow us to help you.

there are multiple ways to debug this. the easiest is to start from scratch. the most skillful one is to set memory watchpoints in a debugger.

Thanks to everyone for their input! I’ve finally found out why there was unexpected behavior with the cv::Mat objects in my program!

After further testing, I found out that the issue only arose when running the program in release mode. In debug mode, the values were not only read properly from file, but also passed correctly to the cv::Mat object.

The issue was that the localization settings for my installed OS were using the DE-standard for numbers, which interprets all periods (.) in numbers as a thousandths separator, NOT a decimal! Thus, the program decided to truncate the fraction part of the double variables.

Going into the localization settings of my PC (locale for me, since I’m currently using Ubuntu,) I changed how the system interpreted numerics, and the problem was solved!

Thanks again for everyone’s input and patience with me!
C.Fox