Aruco module essential functions not implemented in Python in OpenCV 4.10.0

[m.ariuz]

Hello,

I am in the process of using a charuco board for calibration. I am following this tutorial: Using ChArUco boards in OpenCV. Camera calibration and pose estimation. | by Ed Twomey | Medium

Anyone else having the problem that essential functions like: “cv.aruco.interpolateCornersCharuco” and “cv.aruco.interpolateCornersCharuco” seem missing.
Even threw the documentation states an existing Python implementation: OpenCV: Aruco markers, module functionality was moved to objdetect module

I also tried following the official documentation for C++, see OpenCV: Calibration with ArUco and ChArUco.
The ArucoDetector in Python doesnt have the “detectBoard” method. So its also impossible to follow this tutorial in total.
But I guess from a hint in the documentation that the functions used by Medium are deprectated? But in no place marked as “removed”!

I got as far as this: (The boilderplate code was left out!)
charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(DICTIONARY_I_AM_USING)
params = cv.aruco.DetectorParameters()
charuco_board = cv.aruco.CharucoBoard(MY_BOARD_SETTINGS)
detector = cv.aruco.ArucoDetector(charuco_marker_dictionary, params)
marker_corners, marker_ids, rejected_candidates = detector.detectMarkers(im_gray)
So far so good:
Example picture with detected markers:

But then getting the object and image point fails:
object_points_t, image_points_t = charuco_board.matchImagePoints( marker_corners, marker_ids)

ERROR:
error: OpenCV(4.10.0) D:\a\opencv-python\opencv-python\opencv\modules\objdetect\src\aruco\aruco_board.cpp:451: error: (-215:Assertion failed) (int)detectedCharucoVecMat[i].total() * detectedCharucoVecMat[i].channels() == 2 in function ‘cv::aruco::CharucoBoardImpl::matchImagePoints’

Any help or working code would be highly appreciated.

P.S.:
My output of the “detectMarkers” method seems valid. Detected corners are std::vector<std::vector<Point2f>t.
(So translated to python an Array of Arrays containing the 4 points made by 2 coordinates each.)
ID’s are std::vector<int> so in Python a list of Integers.
So I guess the python function “matchImagePoints” gets what it wants!
The detection seems succesful.

I already tried changing the corner array: The detectMarkers method returns a tuple. I used the folloqing code to create the desired array of shape (X, 4, 2). (X beeing the number of deteced markers. Each has 4 corners with 2 coordinates x and y.)
marker_corners = np.array(marker_corners)
marker_corners = np.squeeze(marker_corners)

[[[8812. 5445.]
[8830. 5923.]
[8344. 5932.]
[8324. 5452.]]

[[7172. 5469.]
[7184. 5947.]
[6695. 5949.]
[6687. 5476.]]

[[3896. 5481.]
[3885. 5952.]
[3396. 5951.]
[3406. 5483.]]
…

[11]
[27]
[19]
…

Both, Passing the original return into the function and passing my modified array are failing. (Also not using squeeze and inputting a e.g. (42, 1, 4, 2) array fails!)

[crackwitz]

just file an issue on opencv’s github.

[m.ariuz]

Does anyone have working Charuco Code one could share with me? (Adding the OpenCV version number.)

I may file an issue, but I guess I dont have the time to wait till it gets resolved

[berak]

why not follow the c++ tutorial properly, and use a CharucoDetector to detect a Charuco board ?

[crackwitz]

related:

How do to camera calibration using charuco board for OpenCV 4.10.0?

https://stackoverflow.com/staging-ground/79083527

[m.ariuz]

Hello Berak, thank you for your reply.

I am not fluent in C++, so you are right I might have missed something. Thank you for poining that out. So I took another look at the C++ doc but even now I couldn’t get more insights.

Could you kindly tell me the missing portion of my code?

I need this part “translated” in Python: board.matchImagePoints(currentCharucoCorners, currentCharucoIds, currentObjectPoints, currentImagePoints);

For the detecion I am using this:

@m.ariuz

detector = cv.aruco.ArucoDetector(charuco_marker_dictionary, params)
marker_corners, marker_ids, rejected_candidates = detector.detectMarkers(im_gray)

This was the cloesest related python code I could figure out. Did I made a mistake here already.

Because the C++ function detector.detectBoard(image, currentCharucoCorners, currentCharucoIds); also returns the corners and Ids. Thats as far as I get with python too!
(The only difference I may spot is that the C++ function returns a ?Matrix?, because its of type “Mat”, and the python one returns an array of arrays.)

Furthermore I am instanciating a board manually with the settings I took in order to print it. I thought the portion detector.detectBoard(image, currentCharucoCorners, currentCharucoIds); is equivalent to:

@m.ariuz

charuco_board = cv.aruco.CharucoBoard(MY_BOARD_SETTINGS)

[m.ariuz]

Hello Crackwitz,

yes I also posted my question on stackoverflow, because it serves a bigger community. I will keep both in syny, when I have a solution, because I guess that I am not the only one running into that problem.

For justification: You seem to be among the most experianced people in this forum and your answer was: ´

@crackwitz

issue on opencv’s github

so I started searching for a code-example elsewere.

But in order to fullfill my “keep in sync”: Here is the minimal working code example I posted on stackoverflow:

In order to replicate the error I am getting use the following picture and name it “charuco_board.png”

Here is the code:

    import cv2 as cv
    import numpy as np

    image = cv.imread("charuco_board.png")
    im_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
    charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
    charuco_board = cv.aruco.CharucoBoard(
        size=(11, 8),
        squareLength=500,
        markerLength=300,
        dictionary=charuco_marker_dictionary
    )
    params = cv.aruco.DetectorParameters()
    detector = cv.aruco.ArucoDetector(charuco_marker_dictionary, params)
    marker_corners, marker_ids, rejected_candidates = detector.detectMarkers(im_gray)
    marker_corners = np.array(marker_corners)
    marker_corners = np.squeeze(marker_corners)
    print(marker_corners, marker_ids)
    try:
        object_points_t, image_points_t = charuco_board.matchImagePoints(
            marker_corners,
            marker_ids
        )
    except cv.error as err:
        print(err)

[m.ariuz]

Hello Berak, thank you for your Insight. At first I want to say that I still didnt managed to get it working!

I checked my code again and you have been right that there is also a cv.aruco.CharucoDetector method! At first I kinda felt ashamed for that embarassing mistake.

Then I checked the code, but it seems that it returns the same as my initial code:

Minimum working code:

import cv2 as cv
import numpy as np

image = cv.imread(“charuco_board.png”)
im_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(11, 8),
squareLength=500,
markerLength=300,
dictionary=charuco_marker_dictionary
)
detector = cv.aruco.CharucoDetector(charuco_board)
result = detector.detectBoard(im_gray)

Printout of result:
A tuple of:

result = (
    None,
    None,
    (
        array([[[8812., 5445.],
                [8830., 5923.],
                [8344., 5932.],
                [8324., 5452.]]], dtype=float32),
        array([[[7172., 5469.],
                [7184., 5947.],
                [6695., 5949.],
                [6687., 5476.]]], dtype=float32),
        ...
    ),
    array([
        [ 3],
        [11],
        [27],
        ...
    ])
)

It returns basically the same, see

@m.ariuz

[[[8812. 5445.]
[8830. 5923.]
[8344. 5932.]
[8324. 5452.]]

(The ID is wrong because I copied it wrong in my initial post.)

Except that the code readability is improved. Thank you for that.

Regarding the C++ doc this needs to follow:

charucoBoard.matchImagePoints(charucoCorners, charucoIds, objPoints, imgPoints);

But:

object_points_t, image_points_t = charuco_board.matchImagePoints(
result[2],
result[3]
)

Still fails with the message:

error: OpenCV(4.10.0) D:\a\opencv-python\opencv-python\opencv\modules\objdetect\src\aruco\aruco_board.cpp:451: error: (-215:Assertion failed) (int)detectedCharucoVecMat[i].total() * detectedCharucoVecMat[i].channels() == 2 in function 'cv::aruco::CharucoBoardImpl::matchImagePoints'

[m.ariuz]

New considerations:

Part 1: Checking the aruco-part in the charuco detection.
First I checked the “aruco” part:

(I just drew the corners and ID’s maually using cv.putText and cv.circle)
Corners are detected in the right order. Furthermore ID’s are all identified correctly!
I also checked for consitancy of the point ordering, threw using rotated pictures of the board. The algorithm behaves as expected and all seems right. If your interested I may add a picture of my test.

I used the following for detection:

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
    size=(11, 8),
    squareLength=500,
    markerLength=300,
    dictionary=charuco_marker_dictionary
)
detector_charuco = cv.aruco.CharucoDetector(charuco_board)
result = detector_charuco.detectBoard(im_gray)
marker_corners_charuco, marker_ids_charuco = result[2:]

But my “original” method returns exactly the same and is therefore equivalent!

So Part 2:
After digging deeper in the documentation I found:
this
Quote: " For cv::CharucoBoard the method expects std::vector<Point2f> or Mat with ChAruco corners (chess board corners matched with Aruco markers)."

It expects this type: List[List[int, int]]
So reshaping the array I got from the aruco detection to this the function doesn’t threw an error anymore.
np.array(marker_corners_charuco).copy().reshape(-1, 2)
but now I got 44 Id’s (because my board has 44 markes) but 4*44=176 corner points. So I had to repeat every ID 4 times. Then passing this two arrays into the function kinda works.

But I ran into two major problems:

First:

The function expects the “chess board corners matched with Aruco markers”. But I got the corners of the aruco markes.
For illustration:
I need the red dots:

Wich function achieves this?

Second:
I guessed that at least the returned object points should be correct, since they are based on the original created pattern. They are marked correctly but some are missing.

Here is a fully functional script to test everything. It even creates the picture for you.

P.S.: One needs to be very careful with the pixel sizes of the used image and margins on the side etc.!

from itertools import cycle
from collections import namedtuple

import cv2 as cv
import numpy as np


CharucoResults = namedtuple("CharucoResults", "marker_corners marker_ids")

SQUARE_LENGTH = 500
MARKER_LENGHT = 300
NUMBER_OF_SQUARES_VERTICALLY = 8
NUMBER_OF_SQUARES_HORIZONTALLY = 11

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY),
squareLength=SQUARE_LENGTH,
markerLength=MARKER_LENGHT,
dictionary=charuco_marker_dictionary
)

image_name = f'ChArUco_Marker_{NUMBER_OF_SQUARES_HORIZONTALLY}x{NUMBER_OF_SQUARES_VERTICALLY}.png'
# Create board picture.
cv.imwrite(
    image_name,
    charuco_board.generateImage(
        [i*SQUARE_LENGTH
         for i in (NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY)])
    )


def detect_charuco(image):
    """Detect the charuco corners in given image.
    """
    detector_charuco = cv.aruco.CharucoDetector(charuco_board)
    result = detector_charuco.detectBoard(im_gray)
    return CharucoResults(*result[2:])


def show_charuco_corners_and_id(image, markers, marker_ids=None):
    """Plot the detected corners for debugging.
    """
    colors = [
        (0, 0, 255), # red
        (0, 255, 0), # green
        (255, 0, 0), # blue
        (255, 0, 255), # something
    ]
    if marker_ids is not None:
        ids = iter(marker_ids)
    color_cycler = cycle(colors)
    for i, marker in enumerate(markers, start=1):
        image = cv.circle(
            image,
            (int(marker[0]), int(marker[1])),
            radius=100,
            color=next(color_cycler),
            thickness=10
        )
        if i%4 == 0 and marker_ids is not None:
            try:
                id = str(next(ids))
            except StopIteration:
                break
            image = cv.putText(
                image,
                id,
                (int(marker[0]), int(marker[1])),
                cv.FONT_HERSHEY_SIMPLEX,
                3,
                (255, 0, 255),
                10,
                cv.LINE_AA
            )
    cv.namedWindow("Markers - detected", cv.WINDOW_NORMAL)
    cv.imshow("Markers - detected", image)
    cv.waitKey(-1)


def repeat_each_element_n_times(input_list, n):
    return [item for item in input_list for _ in range(n)]


image = cv.imread(image_name)
im_gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)

detection_results = detect_charuco(im_gray)
show_charuco_corners_and_id(
    image.copy(),
    np.array(detection_results.marker_corners).copy().reshape(-1, 2),
    detection_results.marker_ids
    )

# Something is wrong here:
object_points_t, image_points_t = charuco_board.matchImagePoints(
    np.array(detection_results.marker_corners).copy().reshape(-1, 2),  # As described in my text.
    np.array(repeat_each_element_n_times(list(detection_results.marker_ids), 4))
)

# Since all markers got detected I would expect it to draw each corner of the
# square sourrounding each aruco marker.
# Also since the image got created in this script it is guaranteed to have
# the appropiate size!
# But it only draws a part of it.
show_charuco_corners_and_id(
    image.copy(),
    object_points_t.copy().reshape(-1, 3)
    )

# Image points seem to be unchanged by the function.
# Also it is missing the "detection" step of the sourrounding checkerboard
# square!
show_charuco_corners_and_id(
    image.copy(),
    image_points_t.copy().reshape(-1, 2)
    )

# Check that the charuco_board class has the right corners.
show_charuco_corners_and_id(
    image.copy(),
    np.array(charuco_board.getObjPoints()).copy().reshape(-1, 3)
    )

[m.ariuz]

I got it working.

You must create a new venv and install pip install opencv-contrib-python. Don’t install pip install opencv-python! Since this removes essential functionalities. It seems that the new objdetect in the non-contrib-opencv is quite buggy.

I am not he only one telling this, see cv2.aruco.calibrateCameraCharuco does not exist · Issue #23493 · opencv/opencv · GitHub

So here a fully standalone working version of test code you can try:
Warning: It fails from time to time. Thats because the homograhies are randomly choosen. In some instances they distort the image so much that no detection may happen. Just restart and try again. (Or cover that “bug” by checking for None.)

from itertools import cycle
from collections import namedtuple
import math

import matplotlib.pyplot as plt
import cv2 as cv
import numpy as np


ArucoResults = namedtuple(
    "ArucoResults",
    [
        "marker_corners",
        "marker_ids",
    ]
)
CharucoCResults = namedtuple(
    "CharucoResults",
    ["number_of_corners",
     "corner_coordinates",
      "corner_id",
    ]
)
CameraCalibrationResults = namedtuple(
    "CameraCalibrationResults",
    [
        "repError",
        "camMatrix",
        "distcoeff",
        "rvecs",
        "tvecs",
    ]
)


SQUARE_LENGTH = 500
MARKER_LENGHT = 300
NUMBER_OF_SQUARES_VERTICALLY = 11
NUMBER_OF_SQUARES_HORIZONTALLY = 8

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY),
squareLength=SQUARE_LENGTH,
markerLength=MARKER_LENGHT,
dictionary=charuco_marker_dictionary
)

image_name = f'ChArUco_Marker_{NUMBER_OF_SQUARES_HORIZONTALLY}x{NUMBER_OF_SQUARES_VERTICALLY}.png'
cv.imwrite(
    image_name,
    charuco_board.generateImage(
        [i*SQUARE_LENGTH
         for i in (NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY)])
    )


def generate_test_images(image):
    """Use random homograpy.

    -> Just to test detection. This doesn't simulate a perspective
    projection of one single camera! (Intrinsics change randomly.)
    For a "camera simulation" one would need to define fixed intrinsics
    and random extrinsics. Then cobine them into a projective matrix.
    And apply this to the Image. -> Also you need to add fixed z
    coordinate to the image (may be randomly choosen in the first place), since a projection is from 3d space into 2d
    space.
    """
    h, w = image.shape[:2]
    src_points = np.float32([[0, 0], [w, 0], [w, h], [0, h]])
    dst_points = np.float32([
        [np.random.uniform(w * -0.2, w * 0.2), np.random.uniform(0, h * 0.2)],
        [np.random.uniform(w * 0.8, w*1.2), np.random.uniform(0, h * 0.6)],
        [np.random.uniform(w * 0.8, w), np.random.uniform(h * 0.8, h)],
        [np.random.uniform(0, w * 0.2), np.random.uniform(h * 0.8, h*1.5)]
    ])
    homography_matrix, _ = cv.findHomography(src_points, dst_points)
    image_projected = cv.warpPerspective(image, homography_matrix, (w, h))
    return image_projected


def detect_charuco(image):
    """Detect the charuco corners in given image.
    """
    detector_charuco = cv.aruco.CharucoDetector(charuco_board)
    result = detector_charuco.detectBoard(image)
    return ArucoResults(*result[2:])


def show_charuco_corners_and_id(image, markers, marker_ids=None):
    """Plot the detected corners for debugging.
    """
    colors = [
        (0, 0, 255), # red
        (0, 255, 0), # green
        (255, 0, 0), # blue
        (255, 0, 255), # something
    ]
    if marker_ids is not None:
        ids = iter(marker_ids)
    color_cycler = cycle(colors)
    for i, marker in enumerate(markers, start=1):
        image = cv.circle(
            image,
            (int(marker[0]), int(marker[1])),
            radius=100,
            color=next(color_cycler),
            thickness=10
        )
        if i%4 == 0 and marker_ids is not None:
            try:
                id = str(next(ids))
            except StopIteration:
                break
            image = cv.putText(
                image,
                id,
                (int(marker[0]), int(marker[1])),
                cv.FONT_HERSHEY_SIMPLEX,
                3,
                (255, 0, 255),
                10,
                cv.LINE_AA
            )
    cv.namedWindow("Markers - detected", cv.WINDOW_NORMAL)
    cv.imshow("Markers - detected", image)
    cv.waitKey(-1)


def display_images(images):
    N = len(images)
    cols = math.ceil(math.sqrt(N))
    rows = math.ceil(N / cols)

    for i, img in enumerate(images):
        plt.subplot(rows, cols, i + 1)
        plt.imshow(img, cmap='gray')
        plt.axis('off')
    plt.tight_layout()
    plt.show()


# Create N test images based on the originaly created pattern.
original_image = cv.imread(image_name)
N = 10
random_images = []
for _ in range(N):
    random_images.append(generate_test_images(original_image))
display_images(random_images)


all_charuco_corners = []
all_charuco_ids = []
for img_bgr in random_images:
    img_gray = cv.cvtColor(img_bgr, cv.COLOR_BGR2GRAY)
    detection_results = detect_charuco(img_gray)
    show_charuco_corners_and_id(
        img_bgr.copy(),
        np.array(detection_results.marker_corners).copy().reshape(-1, 2),
        detection_results.marker_ids
        )
    detected_charuco_corners = CharucoCResults(
        *cv.aruco.interpolateCornersCharuco(
            markerCorners=detection_results.marker_corners,
            markerIds=detection_results.marker_ids,
            image=img_gray,
            board=charuco_board
        )
    )
    show_charuco_corners_and_id(
        img_bgr.copy(),
        detected_charuco_corners.corner_coordinates.copy().reshape(-1, 2),
        )
    all_charuco_corners.append(detected_charuco_corners.corner_coordinates)
    all_charuco_ids.append(detected_charuco_corners.corner_id)

calibration_results = CameraCalibrationResults(
    *cv.aruco.calibrateCameraCharuco(
        all_charuco_corners,
        all_charuco_ids,
        charuco_board,
        img_gray.shape[0:2],
        None,
        None
    )
)

Desired output something like:
Detected Aruco markers:

Detected Charuco corners:

My requirements.txt

asttokens==2.4.1
backcall==0.2.0
colorama==0.4.6
comm==0.2.2
contourpy==1.1.1
cycler==0.12.1
debugpy==1.8.7
decorator==5.1.1
executing==2.1.0
fonttools==4.54.1
importlib_metadata==8.5.0
importlib_resources==6.4.5
ipykernel==6.29.5
ipython==8.12.3
jedi==0.19.1
jupyter_client==8.6.3
jupyter_core==5.7.2
kiwisolver==1.4.7
matplotlib==3.7.5
matplotlib-inline==0.1.7
nest-asyncio==1.6.0
numpy==1.24.4
opencv-contrib-python==4.10.0.84
packaging==24.1
parso==0.8.4
pickleshare==0.7.5
pillow==10.4.0
platformdirs==4.3.6
prompt_toolkit==3.0.48
psutil==6.0.0
pure_eval==0.2.3
Pygments==2.18.0
pyparsing==3.1.4
python-dateutil==2.9.0.post0
pywin32==308
pyzmq==26.2.0
six==1.16.0
stack-data==0.6.3
tornado==6.4.1
traitlets==5.14.3
typing_extensions==4.12.2
wcwidth==0.2.13
zipp==3.20.2

opencv build info:

General configuration for OpenCV 4.10.0 =====================================
  Version control:               4.10.0

  Extra modules:
    Location (extra):            D:/a/opencv-python/opencv-python/opencv_contrib/modules
    Version control (extra):     4.10.0

  Platform:
    Timestamp:                   2024-06-17T18:00:01Z
    Host:                        Windows 10.0.17763 AMD64
    CMake:                       3.24.2
    CMake generator:             Visual Studio 14 2015
    CMake build tool:            MSBuild.exe
    MSVC:                        1900
    Configuration:               Debug Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (16 files):         + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (8 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (36 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2

  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe  (ver 19.0.24247.2)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /O2 /Ob2 /DNDEBUG 
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /Zi /Ob0 /Od /RTC1 
    C Compiler:                  C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /O2 /Ob2 /DNDEBUG 
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /Zi /Ob0 /Od /RTC1 
    Linker flags (Release):      /machine:x64  /NODEFAULTLIB:atlthunk.lib /INCREMENTAL:NO  /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:libcpmtd.lib /NODEFAULTLIB:msvcrtd.lib
    Linker flags (Debug):        /machine:x64  /NODEFAULTLIB:atlthunk.lib /debug /INCREMENTAL  /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcpmt.lib /NODEFAULTLIB:msvcrt.lib
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          wsock32 comctl32 gdi32 ole32 setupapi ws2_32
    3rdparty dependencies:       libprotobuf ade ittnotify libjpeg-turbo libwebp libpng libtiff libopenjp2 IlmImf zlib ippiw ippicv

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape signal stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    java world
    Disabled by dependency:      -
    Unavailable:                 alphamat cannops cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv freetype hdf julia matlab ovis python2 sfm ts viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  Windows RT support:            NO

  GUI:                           WIN32UI
    Win32 UI:                    YES
    VTK support:                 NO

  Media I/O: 
    ZLib:                        build (ver 1.3.1)
    JPEG:                        build-libjpeg-turbo (ver 3.0.3-70)
      SIMD Support Request:      YES
      SIMD Support:              NO
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         build (ver 1.6.43)
      SIMD Support Request:      YES
      SIMD Support:              YES (Intel SSE)
    TIFF:                        build (ver 42 - 4.6.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES

  Parallel framework:            Concurrency

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2021.11.0 [2021.11.0]
           at:                   D:/a/opencv-python/opencv-python/_skbuild/win-amd64-3.9/cmake-build/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2021.11.0)
              at:                D:/a/opencv-python/opencv-python/_skbuild/win-amd64-3.9/cmake-build/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)

  OpenCL:                        YES (NVD3D11)
    Include path:                D:/a/opencv-python/opencv-python/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/hostedtoolcache/windows/Python/3.9.13/x64/python.exe (ver 3.9.13)
    Libraries:                   C:/hostedtoolcache/windows/Python/3.9.13/x64/libs/python39.lib (ver 3.9.13)
    Limited API:                 YES (ver 0x03060000)
    numpy:                       C:/hostedtoolcache/windows/Python/3.9.13/x64/lib/site-packages/numpy/_core/include (ver 2.0.0)
    install path:                python/cv2/python-3

  Python (for build):            C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe

  Java:                          
    ant:                         NO
    Java:                        YES (ver 1.8.0.412)
    JNI:                         C:/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.412-8/x64/include C:/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.412-8/x64/include/win32 C:/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.412-8/x64/include
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    D:/a/opencv-python/opencv-python/_skbuild/win-amd64-3.9/cmake-install
-----------------------------------------------------------------

[m.ariuz]

So my final python-version working without the contrib library:
This took me definitly way too long.

So the final issue: Be careful, when setting up the board via cv.aruco.CharucoBoard()! This needs the right setting. Especially switching the row and column size from e.g. (11, 8) to (8, 11) may result in problems.

Standalone code:

from typing import NamedTuple
import math

import matplotlib.pyplot as plt
import cv2 as cv
import numpy as np

class BoardDetectionResults(NamedTuple):
    charuco_corners: np.ndarray
    charuco_ids: np.ndarray
    aruco_corners: np.ndarray
    aruco_ids: np.ndarray


class PointReferences(NamedTuple):
    object_points: np.ndarray
    image_points: np.ndarray


class CameraCalibrationResults(NamedTuple):
    repError: float
    camMatrix: np.ndarray
    distcoeff: np.ndarray
    rvecs: np.ndarray
    tvecs: np.ndarray


SQUARE_LENGTH = 500
MARKER_LENGHT = 300
NUMBER_OF_SQUARES_VERTICALLY = 11
NUMBER_OF_SQUARES_HORIZONTALLY = 8

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY),
squareLength=SQUARE_LENGTH,
markerLength=MARKER_LENGHT,
dictionary=charuco_marker_dictionary
)

image_name = f'ChArUco_Marker_{NUMBER_OF_SQUARES_HORIZONTALLY}x{NUMBER_OF_SQUARES_VERTICALLY}.png'
charuco_board_image = charuco_board.generateImage(
        [i*SQUARE_LENGTH
         for i in (NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY)]
)
cv.imwrite(image_name, charuco_board_image)


def plot_results(image_of_board, original_board, detection_results, point_references):
    fig, axes = plt.subplots(2, 2)
    axes = axes.flatten()
    img_rgb = cv.cvtColor(img_bgr, cv.COLOR_BGR2RGB)
    axes[0].imshow(img_rgb)
    axes[0].axis("off")

    axes[1].imshow(img_rgb)
    axes[1].axis("off")
    axes[1].scatter(
        np.array(detection_results.aruco_corners).squeeze().reshape(-1, 2)[:, 0],
        np.array(detection_results.aruco_corners).squeeze().reshape(-1, 2)[:, 1],
        s=5,
        c="green",
        marker="x",
    )
    axes[2].imshow(img_rgb)
    axes[2].axis("off")

    axes[2].scatter(
        detection_results.charuco_corners.squeeze()[:, 0],
        detection_results.charuco_corners.squeeze()[:, 1],
        s=20,
        edgecolors="red",
        marker="o",
        facecolors="none"
    )
    axes[3].imshow(cv.cvtColor(charuco_board_image, cv.COLOR_BGR2RGB))
    axes[3].scatter(
        point_references.object_points.squeeze()[:, 0],
        point_references.object_points.squeeze()[:, 1]
    )
    fig.tight_layout()
    fig.savefig("test.png", dpi=900)
    plt.show()


def generate_test_images(image):
    """Use random homograpy.

    -> Just to test detection. This doesn't simulate a perspective
    projection of one single camera! (Intrinsics change randomly.)
    For a "camera simulation" one would need to define fixed intrinsics
    and random extrinsics. Then cobine them into a projective matrix.
    And apply this to the Image. -> Also you need to add a random z
    coordinate to the image, since a projection is from 3d space into 2d
    space.
    """
    h, w = image.shape[:2]
    src_points = np.float32([[0, 0], [w, 0], [w, h], [0, h]])
    dst_points = np.float32([
        [np.random.uniform(w * -0.2, w * 0.2), np.random.uniform(0, h * 0.2)],
        [np.random.uniform(w * 0.8, w*1.2), np.random.uniform(0, h * 0.6)],
        [np.random.uniform(w * 0.8, w), np.random.uniform(h * 0.8, h)],
        [np.random.uniform(0, w * 0.2), np.random.uniform(h * 0.8, h*1.5)]
    ])
    homography_matrix, _ = cv.findHomography(src_points, dst_points)
    image_projected = cv.warpPerspective(image, homography_matrix, (w, h))
    return image_projected


def display_images(images):
    N = len(images)
    cols = math.ceil(math.sqrt(N))
    rows = math.ceil(N / cols)

    for i, img in enumerate(images):
        plt.subplot(rows, cols, i + 1)
        plt.imshow(img, cmap='gray')
        plt.axis('off')
    plt.tight_layout()
    plt.show()


# Create N test images based on the originaly created pattern.
N = 10
random_images = []
charuco_board_image = cv.cvtColor(charuco_board_image, cv.COLOR_GRAY2BGR)
for _ in range(N):
    random_images.append(generate_test_images(charuco_board_image))
display_images(random_images)


total_object_points = []
total_image_points = []
for img_bgr in random_images:
    img_gray = cv.cvtColor(img_bgr, cv.COLOR_BGR2GRAY)
  charuco_detector = cv.aruco.CharucoDetector(charuco_board)
  detection_results = BoardDetectionResults(
      *charuco_detector.detectBoard(img_gray)
  )

    point_references = PointReferences(
        *charuco_board.matchImagePoints(
            detection_results.charuco_corners,
            detection_results.charuco_ids
        )
    )
    plot_results(
        img_gray,
        charuco_board_image,
        detection_results,
        point_references
    )
    total_object_points.append(point_references.object_points)
    total_image_points.append(point_references.image_points)


calibration_results = CameraCalibrationResults(
    *cv.calibrateCamera(
        total_object_points,
        total_image_points,
        img_gray.shape,
        None,
        None
    )
)



"""P.S.: Markers are too small in bigger pictures. They seem to not be adjustable.
img_bgr_aruco = cv.aruco.drawDetectedMarkers(
    img_bgr.copy(),
    detection_results.aruco_corners
)
img_bgr_charuco = cv.aruco.drawDetectedCornersCharuco(
    img_bgr.copy(),
    detection_results.charuco_corners
)
"""

Boiled down to the important stuff:

# Setup.
img_gray = "your_favourite_charuco_image.png"

SQUARE_LENGTH = 500
MARKER_LENGHT = 300
NUMBER_OF_SQUARES_VERTICALLY = 11
NUMBER_OF_SQUARES_HORIZONTALLY = 8

charuco_marker_dictionary = cv.aruco.getPredefinedDictionary(cv.aruco.DICT_6X6_250)
charuco_board = cv.aruco.CharucoBoard(
size=(NUMBER_OF_SQUARES_HORIZONTALLY, NUMBER_OF_SQUARES_VERTICALLY),
squareLength=SQUARE_LENGTH,
markerLength=MARKER_LENGHT,
dictionary=charuco_marker_dictionary
)

# Board detection.
charuco_detector = cv.aruco.CharucoDetector(charuco_board)
detection_results = charuco_detector.detectBoard(img_gray)
)

# Getting object and image points.
object_points, image_points = charuco_board.matchImagePoints(
        detection_results[0],
        detection_results[1]
)