Aruco Projection positioning

C++
,
1

Hi! I have a micro robot with about height of 2.5cm and radius of 4cm.
On the top of the robot, I put a Aruco mark on it, and also at the bottom, there are two photoresistors which are very close to the ground.

From
cv::aruco::estimatePoseSingleMarkers(corners, length, cameraMatrix, distCoeffs, rvecs, tvecs)

I get rotation vector rvecs and translation vector tvecs of the maker.
However tvecs is translation vector of the center of maker (in world coordinate).

Then,I use

axesPoints.push_back(Point3f(left_x, left_y,0))
axesPoints.push_back(Point3f(right_x, right_y,0))

Here left_x and left_y are X- and Y- axis coordinate of left photoresistor about center of maker, and right_x and right_y are coordinates of right photoresistor.

I use
cv::projectPoints(axesPoints, rvecs, tvecs, camera_matrix, distortion_coefficients, imagePoints)

to get the coordinates of two photoresistors in the image/pixel coordinate system. I can get
accurate coordinates of the two photoresistors just when the marker is directly below the camera, but in other positions, it’s a little bad.
Even if I take into account the height difference between the photoresistors and the marker,

axesPoints.push_back(Point3f(left_x, left_y,h))
axesPoints.push_back(Point3f(right_x, right_y,h))

it did not improve.

If I can get the translation and rotation of the plane where the photoresistors are located instead of the marker plane, I will get the exact coordinates of the photoresistors in the pixel coordinate system. However I don’t konw how to do it.

Or how can I get the accurate coordinates in the image/pixel coordinate system of two photoresistors by the maker on the top of robot.

Any hints would be appreciated
Best regards.

2

welcome

please post data (pictures).

work on your code so it runs in realtime, i.e. with a video feed. observe how the projected points behave while you move the camera. perhaps post such a video. can you see a pattern?

verify that an object-frame vector of (0,0,0) behaves correctly. give the object-frame coordinates of the marker’s corners, see if they get mapped correctly.

3

Thank you very much for your reply!

A camera is fixed on the roof, a TV below the camera. So I can only move the robot .
Here is a video in realtime when I move the robot.
video1

As shown in the video1, from the center to the edge of the TV, the positioning of the photoresistor is
more and more deviated from the center of the projection circle.

4

In the video1, I slowly moved the robot from directly below the camera to its lower right, that means
from the center of the TV to the edge of the TV, as shown in the picture.

fig1
fig11293×729 20.1 KB

How to make the photoresistors always in the center of the projection circle.

5

As shown in the video2, when I place the marker directly on the TV and move it from the centre of the TV to the edge, the projection positioning does not change, i.e. the photoresistor does not move further and further away from the centre of the projection circle. So I can ensure the object-frame vector of (0,0,0) behaves correctly.
video2

For the robot, I suspect that the projection error comes from the height difference between the photoresistors and the marker, because ·tvecs· and· rves· are translation and rotation vector of the center of maker, not the plane in which the photoresistors are located.

We look forward to your reply.
Best regards!

6

you need adjustments to the objectPoints (first) argument to projectPoints.

projectPoints takes a single rvec and tvec, so please call those arguments that, rather than a plural (“rvecs”, “tvecs”).

documentation says that the “world” coordinate system is the marker’s coordinate system (frame), and rvec&tvec encode the transformation into camera frame.

so all you need is to put points with a negative z into objectPoints… assuming the marker’s frame has x,y in its plane and z being a normal vector away from it.

do this: draw a “box” into your camera view (and show that view)

https://docs.opencv.org/master/d7/d53/tutorial_py_pose.html

7

Thank you! I tried, but it didn’t improve.

so all you need is to put points with a negative z into objectPoints… assuming the marker’s frame has x,y in its plane and z being a normal vector away from it.

However, by using geometric optics rather than the aruco library, I have basically solved this problem, making the projection positioning more accurate, which meets my requirements.