Placing the image on the side of the face

Python
, ,
1

Hello,
I’m trying to use opencv in python to create something like an instagram filter where I can try different types of glasses to fit my face.
I was able to detect the eyes using the mediapipe library and place glasses on their coordinates using the bitwise_and and add functions.
Now I would like to add legs to the glasses, but I don’t know how I should place the image of the legs on the side of my face.

  1. If I use the same method as for the glasses, it looks like legs are sticking out of my ear: D (red marked area in the picture)
  2. I tried to perform the transformation using homography and then use the warpPerspective function, but I get the same result as in the previous point
  3. It occurred to me that I would only show the legs if I detect the side of the face, but I don’t know how to detect the rotation of the face to the side

I will be happy for any advice or idea. Thank you

Snímek obrazovky 2022-03-31 143224
Snímek obrazovky 2022-03-31 143224808×641 27.2 KB

2

you would need to recover the 3D pose of the head

then you could draw a leg (the entire glasses) using the proper perspective.

you’d also want something like “culling”, so you don’t draw the leg if it is hidden by the glasses themselves (or the head, but you don’t have the 3d shape of the head)