Hi everyone!
I am currently working on a software to track real-world objects and place an equivalent 3D version on top of it. For the past 6 months i have extensively researched and developed multiple versions of the software but i am now facing a problem i cannot solve nor find proper articles about it.
To summarize, the current state of the software is able to detect objects similar to what is done in this article OpenCV: Real Time pose estimation of a textured object
Which uses a combination of ORB pattern detection and SolvePnP to extract the object’s pose from image frame key features. But the problem is that it only works for planar objects, anything that is more complex, the software no longer can extract the correct pose and alignment.
I tried the article’s method of getting the pose of the object, with the limitation of only working with planar objects. I also tried going with an ICP (iterative closest point) approach but point-clouds are only generated with the camera movement and i need a way of tracking with a fixed camera and moving object. I have started to look into POSIT but i don’t know if it’s a good option.
What am i missing in terms of implementations, how can i achieve complex geometry / non-planar pose tracking taking into consideration the article mentioned? And how would one keep the pose from one angle to another (360 view of an object)? I would really appreciate some guidance in this matter!
I will always have either a point-cloud of the object or a 3D model in which i can match 2D-points with vertices of the model or points of the cloud itself.
The 2D-3D correspondence can be achieved manually or automatically but these correspondences matching the keyfeatures found in the current frame i don’t know how to tackle.
The ultimate goal would be to recreate as close as possible this type of object tracking https://www.youtube.com/watch?v=D60Q18lMTz8
Is solvepnp the correct path?
Cheers!
