SolvePNP on rectangle giving pose glitches

Thanks for your answer.

I want highlight that this is not happening in general but only when marker is “looking” at the camera.

I’d appreciate some info how I can determine when this situation might happen. I was trying different ways but nothing was working so far.

Greg