Well, I’m not sure if I should be happy or disappointed. On one hand I can make solvePnPRansac return a pretty coherent result, but on the other hand it is very noisy and moves a lot even when not touching the drill.
So the point here is to use an orange marker to know which direction the drill is facing, and use the aligned green markers to detect 3 dot lines (or 2 dot lines if 1 marker is not extracted properly, it still works). Using the orange marker position and the direction of the drill I can assume where the top is (red line) using the angle from the centroid (green dot). My algorithm is not perfect yet, but the relations between 2D and 3D points are correct.
Here are 2 different results from a video capture (I did not move the drill between the screenshots) :
I’ll keep working on this to improve pose estimation.