How to zoom on mouth only in face?

When I only detect the face the camera zoom on the face and follows it. But when I want to detect & zoom/follow the mouth it doesn’t. It just randomly detects all type of stuff expect the mouth.

So for the mouth detection to work I need to detect the face first before I can detect the mouth succesfully.

So this works(only face):

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
face = face_cascade.detectMultiScale(gray, scaleFactor=SCALE_FACTOR, minNeighbors=MIN_NEIGHBORS, minSize=MINSIZE, )
boxes = np.array(face)

But this doesn’t(first face then mouth):

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
face = face_cascade.detectMultiScale(gray, 1.3, 15)
for (x, y, w, h) in face:
roi_gray = gray[y:y + w, x:x + w]
mouth = mouth_cascade.detectMultiScale(roi_gray, scaleFactor=SCALE_FACTOR, minNeighbors=MIN_NEIGHBORS, minSize=MINSIZE,)
boxes = np.array(mouth)

What is the problem? How do I make this work?? How will I be able to detect the mouth and zoom/follow it? Any help or tips are welcome!

Files:

Example(Instead of focusing on the face it should focus on the mouth)

for starters, there’s a simple typo:

you want

roi_gray = gray[y:y + h, x:x + w]

(wont change much, since most face cascades are trained on quadratic rects)

also, you should not use the same MINSIZE to find eyes and faces, think of it.

in the end, the cascades approach is quite brittle, we got better tools nowadays,
e.g try the dnn based face detection here
(which also comes with builtin landmarks to detect the mouth !)

Thank you I’ll fix the type and have a look!

dont forget, that the eyes in the images might be smaller than the minsize for the faces, and thus be rejected

related:

Thanks! You’ve been a help :wink: