Non-planar camera calibration returns system error

Python
1

Hello,

i’m attempting a camera calibration of a non-planar marker collection but receive this error:

SystemError: <built-in function calibrateCameraAruco> returned NULL without setting an error

I know this is not much info. I’m calling the function like this:

ret, mtx, dist, rvecs, tvecs, stdDeviationsIntrinsics, stdDeviationsExtrinsics, perViewErrors = aruco.calibrateCameraAruco(pattern.ArucoCorners, pattern.ArucoIds, counter, pattern.Board.Board, pattern.ImgRes, mtxInit, distInit, flags, criteria)

with cv2.CALIB_USE_INTRINSIC_GUESS as a flag and (cv2.TERM_CRITERIA_EPS & cv2.TERM_CRITERIA_COUNT, 10000, 1e-9) as criteria.

The number of markers in a image frame is between 10 and 400 atm…
Any hints would be greatly appreciated.
Thanks
Markus

2

Mya be you can share values corners ids count and board and imagesize

3

Hi @laurent.berger

I’m attaching these things as a link to the files below in a gist.

  • image resolution is: 1920x1080
  • marker dict is DICT_5X5_1000
  • marker size is 146 px
  • marker separation is 30 px
  • the found marker count is 12

the text files include:

  • initalMtx
  • objPts
  • imgPts
  • ids

test values for aruco calibrate · GitHub

Thank you very much!
Markus

4

How many images do you use to calibrate?
Can you show your grid?

5

Hi @laurent.berger ,

that might be a hint. I tried running the calibration to test my code after capturing a single image from a simulation.

Screenshot below:
image

I’ll capture more and try to run it again.

Thanks
Markus

6

Hi @laurent.berger ,

taking multiple snapshots, I’m getting the same error.

cheers
Markus

7

how do you create pattern.Board.Board?

8

Hi @laurent.berger ,

to describe this a bit more - I’m working in a visual programming environment TouchDesigner with python as a scripting engine. For a project I have created 3 components:

  • a board generator
  • a pattern detector
  • a camera calibrator

The board creates a class depending on board type (checker, circle, charuco, aruco) with information on the board configuration and the actual aruco board saved as a member .Board
The pattern detector finds the pattern and moves all relevant information into a pattern class. Part of that class is also a .Board member which get’s the Board’s class assigned.
Hence in the camera calibrator, the Board is fetched via pattern.Board.Board

the board itself is created such with mrkObjPts being the same that are posted in the gist and the mrkIds a list from 0 to 444:

library = aruco.DICT_5X5_1000
aruDict = aruco.getPredefinedDictionary(library)
aruco.Board_create(mrkObjPt, aruDict, mrkIds)

My next steps to move further would be to try to get this going with the regular calibrateCamera. I can fish out the objPts and imgPts necessary with the found ids, so that shouldn’t be a problem. Still unclear though what I might be doing wrong.

I can’t share the whole code as it’s distributed across multiple classes but perhaps I should write a quick testcase in a single script…

Thanks again
Markus

9

I create my charucoboard like this at the beginning of the program :

class ParamMire:
    def __init__(self):
        self.nb_lig = 6
        self.nb_col = 9
        self.nb_lig_aruco = 5
        self.nb_col_aruco = 8
        self.dim_carre = 0.0275
        self.dim_aruco = 0.03455
        self.sep_aruco = 0.02164
        self.dict = cv.aruco.DICT_5X5_1000
        self.dictionary = cv.aruco.Dictionary_get(self.dict)
        self.board =	cv.aruco.CharucoBoard_create(self.nb_col_aruco, self.nb_lig_aruco,
                                                  self.dim_aruco, self.sep_aruco,
                                                  self.dictionary)
        self.detectorParams = cv.aruco.DetectorParameters_create()
        self.detectorParams.cornerRefinementMethod =  cv.aruco.CORNER_REFINE_SUBPIX

full program is here

10

Hi @laurent.berger,

you are creating charuco while I’m creating aruco. I’m guessing hence that everything I do seems correct?
I have no trouble with planar charucos, chessboards, circlegrids etc. it’s only non-planar aruco.

cheers
Markus

11

works for only planar grid?

12

Hi @laurent.berger,

for planar boards you would use the GridBoard Class - The Board class does not have that limitation.

cheers
Markus

13

I still do not understand : “All markers are placed in the same plane in a grid arrangement.” Have you got two planes or one?

14

Hi @laurent.berger

Right this is the limitation of the GridBoard Class for calibrating planar boards. Yet I’m trying to calibrate a non-planar setup hence I’m using the Board Class which does not have the planar limitation.

My setup has an x-amount of planes theoretically and in the testcase 3.

Cheers
Markus

15

ok - this took me a while:

the mistake was in not specifying flags and criteria as keyword arguments

# works:
aruco.calibrateCameraAruco(pattern.ArucoCorners, pattern.ArucoIds, counter, pattern.Board.Board, pattern.ImgRes, mtxInit, distInit, flags=flags, criteria=criteria)
# doesnt work:
aruco.calibrateCameraAruco(pattern.ArucoCorners, pattern.ArucoIds, counter, pattern.Board.Board, pattern.ImgRes, mtxInit, distInit, flags, criteria)

cheers
Markus

1 Like
16

Sorry I didn’t check this.
Note that you don’t need to use calibrateCameraAruco you can use calibrateCamera with your grid. Sometime you can have better results first you calibrate intrinsic and then calibrate distorsion read code here and here

17

Hi @laurent.berger ,

thank you for the tip regarding intrinsics and distortion. Is there a benefit in using calibrateCamera over the aruco version or is aruco just a wrapper around it? I like that I don’t have to fish out the object points for it.

cheers
Markus

18

calibrateCameraAruco call calibrateCamera
I use only calibrateCamera because I use a chessboard and a charuco grid

19

@markus - hello!~

A quick note that you will drop accuracy if you use the Aruco style board vs ChArUco. This is because the subpixel quality of corners is much higher for inner corners (e.g. up 10x, especially for mono cameras). Non-planar calibration objects also introduce errors from fabrication (in my experience the loss of quality of data outweighs the additional value of the non-planar data)

I presume you might be doing some kind of XR stage setup there? If so, also note that corner subpixel quality is also reduced when working with screens vs printouts (e.g. interference between pixel grids of camera and screen).

I hope you manage to get something working well over there.

If you just want the extrinsics between the non-planar board and the camera (i.e. already have cameraMatrix and distortionCoefficients that you trust) e.g. for XR calibration then none of this is really too much of a concern (but in that case you can also use the solvePnP function instead of calibrateCamera)

I’m making a lot of presumptions here so apologies if I’m off the mark

20

Hi @elliot ,

your assumption if fully on point :slight_smile:
Working on a solution for arbitrarily shaped led stages (curved) knowing though that every led panel is planar. Hence the initial idea to use aruco markers to do lens and hand-eye calibration to figure out the tracker to lens transformation.

I already hit a snag with the lens calibration as the error shoots sky high on narrow zooms and your message supports my thought that for this part I have to use a board based (charuco) approach. To gather the information for the hand-eye calibration, I can then use the aruco markers and solvePnP.

Thanks for chiming in!
Best
Markus