The gesture's code with opencv doesn't count the finger

1

Hi, I found the follow code with opencv:

    import cv2
    import numpy as np
    import copy
    import math
    import os

    def calculateFingers(res, drawing):
        #  convexity defect
        hull = cv2.convexHull(res, returnPoints=False)
        if len(hull) > 3:
            defects = cv2.convexityDefects(res, hull)
            if defects is not None:
                cnt = 0
                for i in range(defects.shape[0]):  # calculate the angle
                    s, e, f, d = defects[i][0]
                    start = tuple(res[s][0])
                    end = tuple(res[e][0])
                    far = tuple(res[f][0])
                    a = math.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
                    b = math.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
                    c = math.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
                    angle = math.acos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c))  # cosine theorem
                    if angle <= math.pi / 2:  # angle less than 90 degree, treat as fingers
                        cnt += 1
                        cv2.circle(drawing, far, 8, [211, 84, 0], -1)
                if cnt > 0:
                    return True, cnt+1
                else:
                    return True, 0
        return False, 0

    # Open Camera
    camera = cv2.VideoCapture(0)
    camera.set(10, 200)
    #while True:
    while camera.isOpened():
        #Main Camera
        ret, frame = camera.read()
        frame = cv2.bilateralFilter(frame, 5, 50, 100)  # Smoothing
        frame = cv2.flip(frame, 1)  #Horizontal Flip
        cv2.imshow('original', frame)
           #Background Removal
        bgModel = cv2.createBackgroundSubtractorMOG2(0, 50)
        fgmask = bgModel.apply(frame)
        kernel = np.ones((3, 3), np.uint8)
        fgmask = cv2.erode(fgmask, kernel, iterations=1)
        img = cv2.bitwise_and(frame, frame, mask=fgmask)
       
        # Skin detect and thresholding
        hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
        lower = np.array([0, 48, 80], dtype="uint8")
        upper = np.array([20, 255, 255], dtype="uint8")
        skinMask = cv2.inRange(hsv, lower, upper)
        cv2.imshow('Threshold Hands', skinMask)     # Getting the contours and convex hull
        skinMask1 = copy.deepcopy(skinMask)
        contours, hierarchy = cv2.findContours(skinMask1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
        length = len(contours)
        maxArea = -1
        if length > 0:
            for i in range(length):
                temp = contours[i]
                area = cv2.contourArea(temp)
                if area > maxArea:
                    maxArea = area
                    ci = i
                    res = contours[ci]
            hull = cv2.convexHull(res)
            drawing = np.zeros(img.shape, np.uint8)
            cv2.drawContours(drawing, [res], 0, (0, 255, 0), 2)
            cv2.drawContours(drawing, [hull], 0, (0, 0, 255), 3)

            isFinishCal, cnt = calculateFingers(res, drawing)
            print( "Fingers", cnt)
            cv2.imshow('output', drawing)
            k = cv2.waitKey(10)
        if k == 27:  # press ESC to exit
            break

The code above should count the finger and print the number of that.
The problem is than it was detect always the wrong number of finger seen from the camera.
I inserted the code in a Raspberry pi 3B.
Please help me to understand why the code doesn’t detect the right number of finger.
That code is not mine, but by an indian programmer.

2

Change this and argument:

cv2.imshow('output', frame)
k = cv2.waitKey(10)
if k == 27:  # press ESC to exit
    break
3

I made that replacing above but doesn’t work.
Why? The number of finger was wrong.

4

It doesn’t work. no video. I saw example from link, but it worked well.

5

I tried to put your code row inside the mine, but it doesn’t count well the finger in the video. Why?

6

I got is working. I changed to camera.set(480, 640. I merely got up 4 fingers. Fortunately, I’m using pi 4B. I have to get skin colour in order to get hand gesture counting

7

I’m using pi 3B: the images are slow. Maybe have I to use pi 4B?

8

My apologized. Used same as above. Change value cv2.bilateralFilter(5, 5, 3). Change this to waitKey(0). The output is not streaming. It is still images. You may have to play around values. I got fingers counting up to 5. Don’t forget set lighting. Or change skin colour code. Actually, there is something wrong with your code. I will try pi 3B+. later.

As for now, I will not be using cv2.createBackgroundSubtractorMOG2. for now.

9

that is rubbish advice. VideoCapture::set takes a property as the first argument, a value as the second argument. it does not take width and height as you demonstrated.

I would very much like to see your responses increase in quality rather than quantity.

1 Like
10

It is not rubbish. I been testing webcam , picam 4k on my 4K monitor. I just comment out, and it will worked. I am using picam camera using cv2.VideoCapture to be testing for both pi 3B/4B.

11

Supra, I made your replacement with pi 3B but the images are slow, and the code doesn’t count right the finger (If I show 5 finger it count 11 finger and etc…).
Why please? Can you help me? I m patient less and less.

12

please post a picture that illustrates the issue. I expect to see a hand, contours, and the convexity defects drawn on top.

13

@legacy. Using RPI 3. I modified some extra:

    def calculateFingers(res, drawing):
    #  convexity defect
    cnt = 0 
    hull = cv2.convexHull(res, returnPoints=False)
    if len(hull) > 3:
        defects = cv2.convexityDefects(res, hull)
        if defects is not None:
            #cnt = 0
            for i in range(defects.shape[0]):  # calculate the angle
                s, e, f, d = defects[i][0]
                start = tuple(res[s][0])
                end = tuple(res[e][0])
                far = tuple(res[f][0])
                a = math.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
                b = math.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
                c = math.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
                s = (a+b+c) / 2
                ar = math.sqrt(s * (s-a) * (s-b) * (s-c))
                d=(2*ar) / a
                angle = math.acos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c)) #* 57  # cosine theorem
                if angle <= math.pi / 2 and d >= math.pi / 6:  # angle less than 90 degree, treat as fingers
                    cnt += 1
                    cv2.circle(drawing, far, 8, [211, 84, 0], -1)
            if cnt > 0:
                return True, cnt #+ 1
            else:
                return True, 0
    return False, 0

Here is output:
original

Here is output. I put white paper behind my hand to get good gesture.
output
U may to have play around with values. I post some more code w/out using backgroundsubstractionMOG and w/out using white object such as walls.

14

My issue:

import cv2
import numpy as np
import copy
import math
import os

def calculateFingers(res, drawing):
    #  convexity defect
    cnt = 0
    hull = cv2.convexHull(res, returnPoints=False)
    if len(hull) > 3:
        defects = cv2.convexityDefects(res, hull)
        if defects is not None:
            #cnt = 0
            for i in range(defects.shape[0]):  # calculate the angle
                s, e, f, d = defects[i][0]
                start = tuple(res[s][0])
                end = tuple(res[e][0])
                far = tuple(res[f][0])
                a = math.sqrt((end[0] - start[0]) ** 2 + (end[1] - start[1]) ** 2)
                b = math.sqrt((far[0] - start[0]) ** 2 + (far[1] - start[1]) ** 2)
                c = math.sqrt((end[0] - far[0]) ** 2 + (end[1] - far[1]) ** 2)
                angle = math.acos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c))  # cosine theorem
                if angle <= math.pi / 2 and d >= math.pi / 6:  # angle less than 90 degree, treat as fingers
                    cnt += 1
                    cv2.circle(drawing, far, 8, [211, 84, 0], -1)
            if cnt > 0:
                return True, cnt+1
            else:
                return True, 0
    return False, 0

# Open Camera
camera = cv2.VideoCapture(0)
camera.set(10, 200)
#while True:
while camera.isOpened():
    #Main Camera
    ret, frame = camera.read()
    frame = cv2.bilateralFilter(frame, 5, 5, 3)  # Smoothing
    frame = cv2.flip(frame, 1)  #Horizontal Flip
    cv2.imshow('original', frame)
       #Background Removal
    bgModel = cv2.createBackgroundSubtractorMOG2(0, 50)
    fgmask = bgModel.apply(frame)
    kernel = np.ones((3, 3), np.uint8)
    fgmask = cv2.erode(fgmask, kernel, iterations=1)
    img = cv2.bitwise_and(frame, frame, mask=fgmask)
    
    # Skin detect and thresholding
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    lower = np.array([0, 48, 80], dtype="uint8")
    upper = np.array([20, 255, 255], dtype="uint8")
    skinMask = cv2.inRange(hsv, lower, upper)
    cv2.imshow('Threshold Hands', skinMask)     # Getting the contours and convex hull
    skinMask1 = copy.deepcopy(skinMask)
    contours, hierarchy = cv2.findContours(skinMask1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    length = len(contours)
    maxArea = -1
    if length > 0:
        for i in range(length):
            temp = contours[i]
            area = cv2.contourArea(temp)
            if area > maxArea:
                maxArea = area
                ci = i
                res = contours[ci]
        hull = cv2.convexHull(res)
        drawing = np.zeros(img.shape, np.uint8)
        cv2.drawContours(drawing, [res], 0, (0, 255, 0), 2)
        cv2.drawContours(drawing, [hull], 0, (0, 0, 255), 3)

        isFinishCal, cnt = calculateFingers(res, drawing)
        print( "Fingers", cnt)
        cv2.imshow('output', frame)
        k = cv2.waitKey(0)
    if k == 27:  # press ESC to exit
        break

The photos of my hand:

IMG_20210204_074238
IMG_20210204_0742384160Ă—3120 1.91 MB

Tell me about that photos and issue.

15

Now the second photo:

IMG_20210204_074302
IMG_20210204_0743024160Ă—3120 1.4 MB

16

The third:

IMG_20210204_074320
IMG_20210204_0743204160Ă—3120 954 KB

17

You’ve forgotten to put white cardboard or white paper behind hand in order to get output result. Do not put ur hand near to webcam… about more than 2 ft. than fiine.

Btw. Do not take photo from camera. U can do c2.imwrite to save filenames.
Here is codoe:

        isFinishCal, cnt = calculateFingers(res, drawing)
        print( f"Fingers", cnt)
        cv2.imwrite('Threshold Hands.jpg', skinMask)
        cv2.imwrite('original.jpg', frame)
        cv2.imwrite('output.jpg', drawing)
        cv2.imshow('output',drawing)
    k = cv2.waitKey(0)
    if k == 27:  # press ESC to exit
        break