Aruco detector doesn't provide subpixel accuracy

,
1

Aruco detector not providing subpixel accuracy

As an example example, I generated this 3x2 Charuco board and annotated the fiducials, I get the following positions for each corner (order of fiducials is provided by Aruco detector opencv method)

10  60 
39  60 
39  89 
10  89 
60  10 
89  10 
89  39 
60  39

However the correct positions are

 9.5  59.5
39.5  59.5
39.5  89.5
 9.5  89.5
59.5   9.5
89.5   9.5
89.5  39.5
59.5  39.5

I understand that for plotting it is required to round to the nearest integer, but still Aruco detector return floating point values without the decimal part (i.e 60.0). Also, it is not a matter of adding 0.5 to every coordinate since some of them require +0.5, and other -0.5. I tried Apriltag corner refinement but seems to be doing alright, not sure if needs additional parameters.

I would appreciate if you could help me with that, cause this is not even a real life picture, this is the cleanest case where the accuracy by default should be optimal

The code to generate the image is

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt

# Generate 
num_squares_x = 3
num_squares_y = 2
square_width_m = 5 
marker_width_m = 3
dictionary_id = cv.aruco.DICT_6X6_250
dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
board = cv.aruco.CharucoBoard((num_squares_x, num_squares_y), square_width_m, marker_width_m, dictionary)
image = board.generateImage((num_squares_x*50, num_squares_y*50)) # this will generate 150x100

# Detect
aruco_params = cv.aruco.DetectorParameters()
aruco_detector = cv.aruco.ArucoDetector(dictionary, aruco_params)
marker_corners, marker_ids, rejected_ids = aruco_detector.detectMarkers(image)

# Annotate (using cross marker)
image_annotated = cv.cvtColor(image, cv.COLOR_GRAY2BGR)
id = 0 
criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 20, 0.001)
for corners in marker_corners:
    for corner in corners[0]:
        curr_position = (int(corner[0]), int(corner[1]))
        cv.drawMarker(image_annotated, position=curr_position, color=(255,0,0), markerType=cv.MARKER_CROSS, markerSize=3, thickness=1)  # Red filled circles
        cv.putText(image_annotated, str(id), (curr_position[0]+2,curr_position[1]+2), cv.FONT_HERSHEY_SIMPLEX, fontScale=0.3, color=(0,255,0), thickness = 1)
        id += 1
    
plt.imshow(image_annotated)
plt.show()

image
image1920×1315 57.5 KB

2

thanks for this. I always suspected there was some carelessness in its implementation but that would confirm it.

this may stem from OpenCV’s “legacy” behavior concerning contours. the contour is the (filled) polygon that would have to be drawn to reproduce the shape. this means the lines are defined to be on and going through the centers of the edge pixels, rather than perfectly delineating the shape (drawing along pixel edges).

feel free to open an issue on OpenCV’s github. if you do, I’d appreciate an @ over there.

oh, you could play around with some of the detection “flavors”. I think there’s a detection method related to “april” tags. one of those is supposed to take all edge pixels, fit lines to those, then calculate the corners from that. maybe that will do better.