I have an image captured by a camera (CAM_1). There is an object of interest at the center of the image (the rest of the image can be considered blacked out). I need to find out how this object would look from a another camera (CAM_2) which is at a small transformation from camera 1. Both cameras have same intrinsics, K. The transformation [R|t] is known too.
My approach is as below. Kindly validate the derivation.
is the object (on) a flat plane?
An example would be a chair on the floor plane. The object(chair) itself is not flat/planar.
for synthetic viewpoint changes, you need a faithful model of the scene.
if the scene were 2D (any flat plane with texture on it), with a little additional information, that could be reconstructed from a single 2D picture.
a 3D scene however can’t be reconstructed from a single 2D picture. you will have to pick a 2D plane in the 3D scene and only for that plane will the mapping be true. the further away from the plane some point lies, the more wrong it will be mapped.
this is a fundamental fact of computer vision.
please find teaching materials that explain this.