Get X, Y, Z world coordinates from 2D Image where Z is always 0

Python
1

Hello everyone,

i came across a problem, that i unfortunately can’t seem to find a solution for.

In a Simulator i have a Camera looking along the x-axis. it is positioned in x =0, y=0, z =1.7 meters world coordinates. (camera specs: Image_width: 1280 px, height: 720px, fov = 100°) I now want to calculate the position of certain pixels from the image in real world coordinates. The z coordinates of the pixel points in world coordinates are always zero. Imagine it like shooting a ray through every pixel and calculate the x and y distance of the point where the ray collides with the ground on z = 0. I know how to create the intrinsic Matrix K and the extrinsic matrix R t. But how can i use these information to get the correct coordinates?

Looking forward to a helping hand.
Thank you,
Luk

2

ray-plane intersection.

get your rays. that uses K, to have them in the camera frame.

then use the pose matrix to get them into the world frame.

then figure out where each ray intersects the ground plane. unless the ray is parallel to it, it’ll intersect. that involves solving a fairly simple equation, or going at it intuitively/geometrically.