Aruco coordinate system changed in pose estimation for OpenCV 4.6/4.7; bug or change in spec?

Python
, ,
1

Hello, fellow OpenCV users!

I recently realized a change in the behavior of the recent Aruco module in OpenCV and I am not sure if that’s a bug or something that was done on purpose (change of specification). I was unable to find any documentation or other bug reports about this, so I hope someone here is able to shed some light on this issue.

I generated a Charuco board of 5x7 squares from a DICT_6x6_250 dictionary, printed the board and took 10 images with my webcam. Now I used the standard procedure of detecting markers (detectMarkers, refineDetectedMarkers, interpolateCornersCharuco) and calibrating the camera (calibrateCameraCharuco). Finally I generated an image with Charuco corners and axes drawn on it using the calibration. With OpenCV version 4.2.0 I got the result on the left. Using the SAME program with OpenCV 4.6.0, I got the result on the right!

cs_42_46
cs_42_461600×600 71.7 KB

The numbering of the corner IDs has changed and that moved the coordinate system from one corner to the other and flipped the Z-axis!

This is not just a drawing issue. The estimated transformation between board and camera has changed, too.

The Charuco board generation has not changed, all generated Charuco boards of OpenCV 4.2.0, 4.6.0 and 4.7.0 are the same, so this is not the issue.

The change has also affected the single Aruco marker coordinate systems. In the current documentation under

https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html

(scroll down to first image in “Pose Estimation”), the text still says “The marker coordinate system that is assumed by this function is placed in the center (by default) or in the top left corner of the marker with the Z axis pointing out, as in the following image.”, but the image shows Z-axes pointing in.

The issue is the same for OpenCV 4.7.0. The interface to the Aruco Library has changed. Adapting my program to OpenCV 4.7.0, the result is the same as for OpenCV 4.6.0 shown above.

I did not upload my code and the full set of data, since I don’t think the issue is with my code. But I can provide it on request.

Thanks in advance for some help.

Cheers, Philipp

2

there is no spec. someone involved with OpenCV/Intel/
Xperience.AI/… decided to change a lot of things in the aruco module, seemingly in preparation for it moving from contrib to the main repo.

that work broke a lot of fundamental assumptions and only began discussing changes for backwards-compatibility after users of the new release(s) noticed and complained.

I don’t know if the situation has been resolved yet. version release notes don’t say much about this. in december of 2022, the aruco module was moved into the main repo, now lives in the “objdetect” module for some reason. breaking changes to the contrib aruco module happened even before that time, possibly as early as november 2021.

please feel free to browse the issues and PRs on OpenCV’s github. if there is anything you’d like improved with the docs or the APIs, let them know and don’t let up.

I believe release version 4.5.4 hasn’t been messed with yet, but 4.5.5 may have. you don’t have to go as far back as 4.2.

there are positive changes in all of this. I see commits and merges that indicate improvements in detection speed and more.

3

The title of my issue suggests that I thought of two reasons for the problem: 1) code and spec have been changed without me knowing or 2) code and spec are not in sync, so there is a bug. But you found another reason: There is no spec, so no-one can tell if its a bug or not! :grin:

The way of changing things in OpenCV without announcing the changes brought me some trouble too. I teach a 3D computer vision class at our local university. The software I wanted to use in class worked on my computer, but students with different versions of OpenCV got different results. Took me some time to find the cause…

I will try to get in contact with the developers by maybe reporting the issue with the documentation I pointed out. That may clear some things.

Well, thanks a lot for pointing me in the right direction!

1 Like