@markus - hello!~
A quick note that you will drop accuracy if you use the Aruco style board vs ChArUco. This is because the subpixel quality of corners is much higher for inner corners (e.g. up 10x, especially for mono cameras). Non-planar calibration objects also introduce errors from fabrication (in my experience the loss of quality of data outweighs the additional value of the non-planar data)
I presume you might be doing some kind of XR stage setup there? If so, also note that corner subpixel quality is also reduced when working with screens vs printouts (e.g. interference between pixel grids of camera and screen).
I hope you manage to get something working well over there.
If you just want the extrinsics between the non-planar board and the camera (i.e. already have cameraMatrix and distortionCoefficients that you trust) e.g. for XR calibration then none of this is really too much of a concern (but in that case you can also use the solvePnP function instead of calibrateCamera)
I’m making a lot of presumptions here so apologies if I’m off the mark