ColorCorrectionModel issue

Hi all. I’m trying to perform color correction on images with a SG140 color checker. I’m just replicating the tutorial on OpenCV (4.6.0) website that you can find here: OpenCV: Color Correction Model

The color checker detection works flawlessly, I’m getting the RGB color patches correctly, the pixel values are provided to the ColorCorrectionModel instance (specifying COLORCHECKER_DigitalSG as color checker type). After the model is ran, I’m getting the CCM matrix to perform the color conversion. Internally the fitting is performed with a Downhill solver using a specific metric for color distance (CIE2000 as default). My issue is that the colors in the final image are not good. I tried to change the color distance, and by default a gamma linearization is applied, but nothing changed significantly. I also tried to face the problem by solving in closed form the linear system, obtaining another CCM. But again, once the CCM is applied to the image, the resulting colors are not good.

// using cv::ccm
cv::ccm::ColorCorrectionModel model(src,cv::ccm::CONST_COLOR::COLORCHECKER_DigitalSG);
model.setCCM_TYPE(cv::ccm::CCM_3x3);
model.setDistance(cv::ccm::DISTANCE_RGB);
model.setLinear(cv::ccm::LINEARIZATION_GAMMA);
model.setLinearGamma(1.088);
//model.setLinearDegree(3);
//model.setSaturatedThreshold(0, 0.98);
//model.setMaxCount(50000);
model.setEpsilon(1e-6);
model.run();
cv::Mat ccm = model.getCCM();
double loss = model.getLoss();
cv::Mat tmpImageCalibrated;
cvtColor(image, tmpImageCalibrated, cv::COLOR_BGR2RGB);
tmpImageCalibrated.convertTo(tmpImageCalibrated, CV_64F);
const int inp_size = 255;
const int out_size = 255;
tmpImageCalibrated /= inp_size;
cv::Mat tmpOutImageCalibrated = model.infer(tmpImageCalibrated);
tmpOutImageCalibrated *= out_size;
tmpOutImageCalibrated.convertTo(tmpOutImageCalibrated, CV_8UC3);
outImageCalibrated = cv::min(cv::max(tmpOutImageCalibrated, 0), out_size);
cvtColor(outImageCalibrated, outImageCalibrated, cv::COLOR_RGB2BGR);
// closed form
// chartsRGB contains the reference RGB values for SG140 color checker
cv::Mat src = chartsRGB.col(1).clone().reshape(3, chartsRGB.rows/3);
src /= 255.;
cv::Mat trg = getColorReference().clone();
src = src.reshape(1, src.size().height);
trg = trg.reshape(1, trg.size().height);
cv::Mat srcFloat, trgFloat;
src.convertTo(srcFloat, CV_32F);
trg.convertTo(trgFloat, CV_32F);
cv::Mat tmp = (srcFloat.t()*srcFloat).inv()*srcFloat.t();
cv::Mat ccm = tmp*trgFloat;
cv::Mat imageReshaped = image.reshape(1,image.size().height*image.size().width);
imageReshaped.convertTo(imageReshaped, CV_32F);
cv::Mat result = imageReshaped * ccm;
cv::Mat resultR = result.reshape(3, image.size().height);
resultR.convertTo(outImageCalibrated, CV_8UC3);
cv::cvtColor(outImageCalibrated,outImageCalibrated,cv::COLOR_RGB2BGR);

Does anyone have found the same issue? Or am I missing something?
Sorry in advance, I’m not an expert in this particular topic, so feel free to suggest me some tutorial or reference to deepen my knowledge.

Just for the ones that could face a similar issue in future… beware the order of colors. For some reasons, the color checker detector in OpenCV outputs color patches from bottom to top. And moreover the SG140 reference colors (the DigitalSG_LAB_D50_2 matrix in color.cpp, inside mcc module) is saved by columns (therefore from left to right).

Anyway I had success with closed form, but I still can’t get a good fit with the cv::ccm::ColorCorrectionModel.

2 Likes

(OpenCV notoriously has serious issues with ordering of things…)

1 Like

An additional info… While the color ordering of SG140 reference is confirmed, the inverse ordering of color patches detected by cv::mcc::CCheckerDetector I mentioned before is not true. After a couple of tests on different images, and a quick check on the source code in that specific module, I found out that there is a sorting going on in checkerAnalysis function, inside checker_detector.cpp, that I think ends up assigning an “orientation” of color checker based on some kind of similarity between the detected colors and the reference. I should check more deeply to understand the details better, but I suppose that (depending on the color accuracy of input image) this technique can “fail”, at least in the specific case of SG140 color checker.