Aruco module essential functions not implemented in Python in OpenCV 4.10.0

Python
9

New considerations:

Part 1: Checking the aruco-part in the charuco detection.
First I checked the “aruco” part:

image
image1920×1060 221 KB

(I just drew the corners and ID’s maually using cv.putText and cv.circle)
Corners are detected in the right order. Furthermore ID’s are all identified correctly!
I also checked for consitancy of the point ordering, threw using rotated pictures of the board. The algorithm behaves as expected and all seems right. If your interested I may add a picture of my test.

I used the following for detection:

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
    size=(11, 8),
    squareLength=500,
    markerLength=300,
    dictionary=charuco_marker_dictionary
)
detector_charuco = cv.aruco.CharucoDetector(charuco_board)
result = detector_charuco.detectBoard(im_gray)
marker_corners_charuco, marker_ids_charuco = result[2:]

But my “original” method returns exactly the same and is therefore equivalent!

So Part 2:
After digging deeper in the documentation I found:
this
Quote: " For cv::CharucoBoard the method expects std::vector<Point2f> or Mat with ChAruco corners (chess board corners matched with Aruco markers)."

It expects this type: List[List[int, int]]
So reshaping the array I got from the aruco detection to this the function doesn’t threw an error anymore.
np.array(marker_corners_charuco).copy().reshape(-1, 2)
but now I got 44 Id’s (because my board has 44 markes) but 4*44=176 corner points. So I had to repeat every ID 4 times. Then passing this two arrays into the function kinda works.

But I ran into two major problems:

First:

The function expects the “chess board corners matched with Aruco markers”. But I got the corners of the aruco markes.
For illustration:
I need the red dots:

image
image710×608 48.5 KB

Wich function achieves this?

Second:
I guessed that at least the returned object points should be correct, since they are based on the original created pattern. They are marked correctly but some are missing. :confused:

Here is a fully functional script to test everything. It even creates the picture for you.

P.S.: One needs to be very careful with the pixel sizes of the used image and margins on the side etc.!

from itertools import cycle
from collections import namedtuple

import cv2 as cv
import numpy as np


CharucoResults = namedtuple("CharucoResults", "marker_corners marker_ids")

SQUARE_LENGTH = 500
MARKER_LENGHT = 300
NUMBER_OF_SQUARES_VERTICALLY = 8
NUMBER_OF_SQUARES_HORIZONTALLY = 11

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY),
squareLength=SQUARE_LENGTH,
markerLength=MARKER_LENGHT,
dictionary=charuco_marker_dictionary
)

image_name = f'ChArUco_Marker_{NUMBER_OF_SQUARES_HORIZONTALLY}x{NUMBER_OF_SQUARES_VERTICALLY}.png'
# Create board picture.
cv.imwrite(
    image_name,
    charuco_board.generateImage(
        [i*SQUARE_LENGTH
         for i in (NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY)])
    )


def detect_charuco(image):
    """Detect the charuco corners in given image.
    """
    detector_charuco = cv.aruco.CharucoDetector(charuco_board)
    result = detector_charuco.detectBoard(im_gray)
    return CharucoResults(*result[2:])


def show_charuco_corners_and_id(image, markers, marker_ids=None):
    """Plot the detected corners for debugging.
    """
    colors = [
        (0, 0, 255), # red
        (0, 255, 0), # green
        (255, 0, 0), # blue
        (255, 0, 255), # something
    ]
    if marker_ids is not None:
        ids = iter(marker_ids)
    color_cycler = cycle(colors)
    for i, marker in enumerate(markers, start=1):
        image = cv.circle(
            image,
            (int(marker[0]), int(marker[1])),
            radius=100,
            color=next(color_cycler),
            thickness=10
        )
        if i%4 == 0 and marker_ids is not None:
            try:
                id = str(next(ids))
            except StopIteration:
                break
            image = cv.putText(
                image,
                id,
                (int(marker[0]), int(marker[1])),
                cv.FONT_HERSHEY_SIMPLEX,
                3,
                (255, 0, 255),
                10,
                cv.LINE_AA
            )
    cv.namedWindow("Markers - detected", cv.WINDOW_NORMAL)
    cv.imshow("Markers - detected", image)
    cv.waitKey(-1)


def repeat_each_element_n_times(input_list, n):
    return [item for item in input_list for _ in range(n)]


image = cv.imread(image_name)
im_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)

detection_results = detect_charuco(im_gray)
show_charuco_corners_and_id(
    image.copy(),
    np.array(detection_results.marker_corners).copy().reshape(-1, 2),
    detection_results.marker_ids
    )

# Something is wrong here:
object_points_t, image_points_t = charuco_board.matchImagePoints(
    np.array(detection_results.marker_corners).copy().reshape(-1, 2),  # As described in my text.
    np.array(repeat_each_element_n_times(list(detection_results.marker_ids), 4))
)

# Since all markers got detected I would expect it to draw each corner of the
# square sourrounding each aruco marker.
# Also since the image got created in this script it is guaranteed to have
# the appropiate size!
# But it only draws a part of it.
show_charuco_corners_and_id(
    image.copy(),
    object_points_t.copy().reshape(-1, 3)
    )

# Image points seem to be unchanged by the function.
# Also it is missing the "detection" step of the sourrounding checkerboard
# square!
show_charuco_corners_and_id(
    image.copy(),
    image_points_t.copy().reshape(-1, 2)
    )

# Check that the charuco_board class has the right corners.
show_charuco_corners_and_id(
    image.copy(),
    np.array(charuco_board.getObjPoints()).copy().reshape(-1, 3)
    )
1 Like