Hand tracking not working with kinect v2

Python
, ,
1

I didn’t have a webcam to test some hand tracking code on, so I used a Xbox Kinect v2. But Vscode kept spitting out the same error. My Kinect would turn on for a brief second and turn off again.
This is the code

import cv2
import mediapipe as mp
import time
 
cap = cv2.VideoCapture(0)
 
mpHands = mp.solutions.hands
hands = mpHands.Hands()
mpDraw = mp.solutions.drawing_utils
 
pTime = 0
cTime = 0
 
while True:
    success, img = cap.read()
    imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    results = hands.process(imgRGB)
    # print(results.multi_hand_landmarks)
 
    if results.multi_hand_landmarks:
        for handLms in results.multi_hand_landmarks:
            for id, lm in enumerate(handLms.landmark):
                # print(id, lm)
                h, w, c = img.shape
                cx, cy = int(lm.x * w), int(lm.y * h)
                print(id, cx, cy)
                # if id == 4:
                cv2.circle(img, (cx, cy), 15, (255, 0, 255), cv2.FILLED)
 
            mpDraw.draw_landmarks(img, handLms, mpHands.HAND_CONNECTIONS)
 
    cTime = time.time()
    fps = 1 / (cTime - pTime)
    pTime = cTime
 
    cv2.putText(img, str(int(fps)), (10, 70), cv2.FONT_HERSHEY_PLAIN, 3,
                (255, 0, 255), 3)
 
    cv2.imshow("Image", img)
    cv2.waitKey(1)

The error code I got was:-

INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
[ WARN:0] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (376) anonymous-namespace'::SourceReaderCB::OnReadSample videoio(MSMF): OnReadSample() is called with error status: -2147024809 [ WARN:0] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (388) anonymous-namespace’::SourceReaderCB::OnReadSample
videoio(MSMF): async ReadSample() call is failed with error status: -2147024809
[ WARN:1] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (1022) CvCapture_MSMF::grabFrame videoio(MSMF): can’t grab frame. Error: -2147024809
Traceback (most recent call last):
File “c:\Users\haris\Documents\GitHub\Haz3-jolt\handtrackingmodule.py”, line 16, in
imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.5.4-dev) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function ‘cv::cvtColor’

[ WARN:1] global D:\a\opencv-python\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (438) `anonymous-namespace’::SourceReaderCB::~SourceReaderCB terminating async callback

2

there is no kinect support built into opencv by default,
you have to build the opencv libs from src, with the kinect/openni sdk.
then, you can use

cap = cv2.VideoCapture(0, cv2.CAP_OPENNI2)
3

just try CAP_DSHOW. afaik the kinects expose their color feed as regular video.

and why do all you beginners never error-check the VideoCapture creation (assert cap.isOpened()) and the reading (if not success: break)? do that. it’s not optional.

4

related: