How do I improve my reprojection error? Charuco Board Calibration

,
1

Hey Everyone!

Using python with the opencv 4.6 Aruco module. It works pretty well, but I’m finding no matter what I do… I can’t beat a .5~.7 reprojection error on the calibration. These higher reprojection error calibrations must contribute to a little bit of noise… thus I’m trying to see if I can lower it. Seems like some of the best calibrations come in around .2?

We use a Charuco board to calibrate and should have it correctly specified in the settings. Our actual ‘board’ is a piece of 8.5 x 11 paper taped down onto a sheet of plywood. We’ve been use a 12x7 (Squares) of the 5x5 1000 Charuco board. Wasn’t clear to me if there was strategy in what you pick for that… so just went with it.

Our calibration procedure is pretty simple… I can save the code below, but basically we tend to capture up to ~100 frames trying to get the board at all different angles.

Verify that we can see at least 8 Charuco IDs, and 4 Charauco Corners. We then save the frame to be processed later. I was looking for frames with a large amount of tags detected.

I’m wondering if this concept is flawed. Am I generating too much data? Are there other checks I should be running on this? Any advice?

    # Captures Frames  
    #   Takes a frame, and if a valid for calibration frame, 
    #       saves it in the calibration path
    #  2 Versions of a Frame are stored, the raw frame and a drawn on one
    #  Design is to allow you to remove frames you don't think would be good
    def captureFrame(self, image: cv2.Mat) -> bool:
        validFrame = True

        # Ensure Frame Calibration Path Exists
        #  if not, create it, remove prior as well
        if not os.path.exists(FileConfigSource.CALIBRATION_FRAME_PATH):
            os.makedirs(FileConfigSource.CALIBRATION_FRAME_PATH)
        
        # Get image size
        if self._imsize == None:
            self._imsize = (image.shape[0], image.shape[1])

        (corners, ids, rejected) = cv2.aruco.detectMarkers(image, self._aruco_dict, parameters=self._aruco_params)
        
        # Check if enough markers are detected
        if(ids is None or len(ids) < 4):  
            validFrame = False
        
        # verify corners have been detected
        if(len(corners) < 1):
            validFrame = False
        
        # Good so far
        if validFrame:

            # Find Charuco corners
            (retval, charuco_corners, charuco_ids) = cv2.aruco.interpolateCornersCharuco(
                corners, ids, image, self._charuco_board)
            
            # Check for invalid conditions
            if(charuco_ids is None or len(charuco_ids) < 8):
                validFrame = False

            # Verify at least 4 charuco corners
            if(charuco_corners is None or len(charuco_corners) < 4):
                validFrame = False

            # Valid Frame?
            if retval and validFrame:
                # Valid Frame for Calibration!
                validFrame = True

                # Incriment and Log Valid Frame
                self.mCalValidFrame = self.mCalValidFrame + 1
                print("Valid Calibration Frame " + str(self.mCalValidFrame))

                # Frame Name Suffix
                frameName = FileConfigSource.CALIBRATION_FRAME_PATH + "cal_" + str(self.mCalValidFrame) + "_"
                print(" creating " + frameName)

                # Save the Raw Version
                rawFrameName = frameName + "raw.jpg"
                cv2.imwrite(rawFrameName, image)

                # Save a Charuco Marked Version
                markedFrameName = frameName + "marked.jpg"
                cv2.aruco.drawDetectedMarkers(image, corners)
                cv2.aruco.drawDetectedCornersCharuco(image, charuco_corners, charuco_ids)
                cv2.imwrite(markedFrameName, image)

                # wait 1.5 seconds for the next frame
                time.sleep(1.5)
2

crosspost: