How to improve PnP pose estimation?

Python
1

I’m doing pose estimation of an ArUco marker with a OAK-D Pro. I’m using

params.cornerRefinementMethod = cv.aruco.CORNER_REFINE_SUBPIX

detector = cv.aruco.ArucoDetector(aruco_dict, params)
[corners, ids, _] = detector.detectMarkers(self.cvImg)

and

_, rvecs, tvecs = cv.solvePnP(objPoints, corners[i], self.mtx, self.dist)

If i rotate the cam just around it’s y axis the translation vector should stay rather constant. I rotated the cam around 13° and the tvec is changing from
tvecs
[[ 0.01710385]
[-0.02610471]
[ 0.33584111]]
to
[[-0.06666096]
[-0.03059485]
[ 0.31829297]]

this means the error is around 5 cm in the x-axis. Is there a way to further improve the pose estimation? Thanks in advance!

2

and which axis is that, relative to the camera, and why do you think it’s that axis, and why should the translation vector not change from that?

the translation vector describes the marker’s pose, in the camera frame. when applied, this transformation transforms marker-local coordinates into camera-local coordinates.

3

Thanks for your reply. That’s what I mean with y axis rotating the cam around the vertical axis.

image

I thought the translation vector should stay the same if i just rotate the camera without translating it - but that’s just the case in the marker frame but not in the camera frame?

I got confused I thought the tvec are given in the marker frame, so the tvec should indeed stay constant when transforming the tvec into the marker frame (RotationMatrix * tvec)?

What do you mean with when applied?

4

Hi @etigafy, I am observing similar results in my setup with a realsense d435 camera and AprilTag instead of aruco marker. I compute the pose of the marker with solvePnP in camera’s frame and then transform it to a fixed world frame. However, when I rotate the camera I can observe a deviation of upto 30mm in translation of the marker in the fixed frame (similar to your issue). Were you able to to resolve this problem?