Converting 2D pixel points to 3D world points

Python
1

i am a studenta nd i am currently doing my masters in robotcs with AI. i am doing my final project and i am stuck with this math. have pixel points and i want to navigate my robot arm (3DOF)end effecter to the point. i have been trying to convert those pixel points to world points, I was going through the internet and found the article regarding it which I am following , I didn’t understand how they converted pixel points (u,v) to world points.can you please help me with this conversion of image points to world points
ex:
image point [502,185],
world point [5.5,3.9,46.8],

Screenshot 2024-08-27 205746
Screenshot 2024-08-27 205746498×698 9.68 KB

link for the article

2

that article did a whole lot of dancing around.

all you need is a homography from image space to the plane in world space. you get that by putting down a checkerboard or aruco marker or anything like that, and getting its pose, or just its four corners.

if this math is new to you, or even just the concepts, you should talk to some study advisor because this should have been taught in a robotics course of study before you start your final project.

3

Hi There,
Thanks alott for responding
i have did the Homography for these 4 points
imgpoints = np.array([
[176.32127, 122.73755],
[529.60095, 111.33149],
[531.8277, 363.7699],
[185.79271, 372.10544]
], dtype=‘float32’)

Define the corresponding real-world coordinates (object points)

objpoints = np.array([
[0, 0],
[17.5, 0],
[17.5, 12.5],
[0, 12.5].
Homography Matrix:
[[ 4.98024732e-02 -1.89158643e-03 -8.54906683e+00]
[ 1.57535315e-03 4.87933860e-02 -6.26654887e+00]
[ 2.99931890e-05 -8.32520821e-05 1.00000000e+00]].

obtained target point real-world coordinates: (17.50 cm, 0.00 cm)
The average Z depth value from the camera to these points is 40cm.

camera is front-facing
camera and robot arm offset:
the camera is placed behind the robot arm at a distance of 26cm and a height of 12.5cm.

my supervisors are not helping me on this,like I am stuck between the bridge,I can handle the kinematics,can you please guide me on this, I am confused about the coordinate systems and how to feed this the robot coordinate system.