I’m trying to make a simple project of a cube with 6 different aruco markers on each side. The goal is to obtain and display a single translation and rotation vector pair which will always show the center of the cube.
For now I managed to rotate all the rvecs so each recognized aruco axes will match the common cube rotation, which works just fine. Modifying the tvecs turned out to be a bit harder. Lets focus on the top side marker.
I tried to subtract the offset vector from the tvec as following:
rvec, tvec, _ = aruco.estimatePoseSingleMarkers(...) tvec = tvec - [0, -markerSideLength / 2, 0] # Moving -half cube down by Y axis
When looking to the side of the marker the offset looks almost fine (image 1), but when the marker is parallel to the camera plane (image 2) we can see the issue:
I quickly understood that the offset I’m subtracting is moving the tvec along the camera Y axis and not the marker Y axis. This could also be seen by that the marker Y axis should be green and not blue, as on previous screenshot (so I actually need to move it along blue Z).
So I need to move the tvec along the marker axis. I searched for a few days and tried to use the translation matrices and cv2.composeRT, but I was only able to get the same invalid or completely non-working result. I can see I need to somehow mix the marker rotation in to make this work, but I can’t understand how to do that.