SolvePNP on rectangle giving pose glitches

Hello. I am using rectangle marker detection like Aruco and then apply SolvePNP to get position and rotation. What I found is that when marker is pointing/facing directly to the camera (as I suppose) it gives me glitches which you can see on the video. I am using SOLVEPNP_IPPE_SQUARE but I have tried also other parameters and all give same problem.

I would appreciate any tips that can help with this problem or smart way to deal with it.

the uncertainty in the rotation and distance?

that’s a mathematical inevitability of planar objects.

there are no solutions to this.

at best you can filter this but filtering is always a tradeoff. here, a filter would incur delay/lag.

Thanks for your answer.

I want highlight that this is not happening in general but only when marker is “looking” at the camera.

I’d appreciate some info how I can determine when this situation might happen. I was trying different ways but nothing was working so far.


yes, and that is exactly the issue.

avoid precisely this situation, or accept noisy orientation in this situation.

imagine the vanishing points of your square. if you look at it from a good angle, those points are well defined. if you look at it from the top down, those points shoot off to infinity.

conversely, the edges of opposite sides approach being parallel and noise (stare at the pixels at those edges…) starts to dominate that difference, but that difference from “parallel” is what pose recovery uses to estimate rotation.

run this stuff interactively. observe the pose solution, and how much it varies, as you rotate the marker.

Thanks. Thanks. Thanks. Thanks. Thanks.
(20 characters required).