Facial Detection+Recognition-Jetson Nano-OpenCV

,
1

For a Facial detection+recognition project (being implemented on the Jetson nano) using OpenCV, what is the best achievable output video frame rate? Is it possible to get frame rates around 25-30 fps using OpenCV & DLib library?

I have JetPack 4.6.1 and would be using a 1080p USB camera on the Jetson Nano. What would the approx. output video frame rate I can expect?

2

There is no specific answer to your question as it depends on what recognition method your using and what image resolution is being used. Really you just need to code prototypes and benchmark your usage and if you don’t get the desired FPS then you need to start making compromises to reach it such as use 360p vs 1080p, or change recognition methods.

This blog post makes a comparison of DLib and OpenCV DNN for recognition.

1 Like
3

25-30 fps is challenging, 10 fps is ok. Pay attention to the following:

  • it is pretty tricky to put Dlib into operation on Jetson Nano nowadays. I would recommend using OpenCV only. OpenCV has no HOG detector for faces but supports deep learning models.
  • OpenCV with GPU support must be compiled on the board
  • It must be compiled with GSTREAMER support, and a suitable pipeline for video must be used (delay in images is horrible otherwise)
  • using power from a power bank and deep learning models, you must switch Jetson Nano to 5W
    You can inspire at https://www.agentspace.org/kv/lesson8.zip ; see the directory “robot” where there is software for JetBot (NVIDIA open source robot controlled by the Jetson Nano)
1 Like
4

Really appreciate the revert.

I’m unable to get a basic facial recognition code below to run at a respectable FPS. The code below only runs at 1-2FPS, which is why I thought of incorporating DLib.

import face_recognition
import cv2
import os
import pickle
import time
print(cv2.__version__)

fpsReport=0
scaleFactor=.25

Encodings=[]
Names=[]

with open('train.pkl','rb') as f:
    Names=pickle.load(f)
    Encodings=pickle.load(f)

font=cv2.FONT_HERSHEY_SIMPLEX
cam=cv2.VideoCapture(0)   

timeStamp=time.time()
while True:
    _,frame=cam.read()
    frameSmall=cv2.resize(frame,(0,0),fx=scaleFactor,fy=scaleFactor)
    frameRGB=cv2.cvtColor(frameSmall,cv2.COLOR_BGR2RGB)
    facePositions=face_recognition.face_locations(frameRGB,model='CNN')
    allEncodings=face_recognition.face_encodings(frameRGB,facePositions)
    for (top,right,bottom,left),face_encoding in zip(facePositions,allEncodings):
        name='Unknown Person'
        matches=face_recognition.compare_faces(Encodings,face_encoding)
        if True in matches:
            first_match_index=matches.index(True)
            name=Names[first_match_index]
        top=int(top/scaleFactor)
        right=int(right/scaleFactor)
        bottom=int(bottom/scaleFactor)
        left=int(left/scaleFactor)
        cv2.rectangle(frame,(left,top),(right,bottom),(0,0,255),2)
        cv2.putText(frame,name,(left,top-6),font, .75, (0,0,255),2) 
    dt=time.time()-timeStamp
    fps=1/dt
    fpsReport=0.90*fps + 0.1*fps
    #print('fps is: ',round(fpsReport,2))
    timeStamp=time.time()
    cv2.rectangle(frame,(0,0),(100,40),(0,0,255),-1)
    cv2.putText(frame,str(round(fpsReport,1)) + 'fps',(0,25), font, .75, (0,255,255,2))
    cv2.imshow('Picture',frame)
    cv2.moveWindow('Picture',0,0)
    if cv2.waitKey(1)==ord('q'):
        break

cam.release()        
cv2.destroyAllWindows()

Kindly suggest ways I could get output video frame rate to about about atleast 15 fps.

PS-I have OpenCV compiled with GStreamer, Cuda and cuDNN support on my Jetson nano.

5

it is a good idea to avoid face_recognition, however I recommend to use pure OpenCV with GPU support instead of DLib.
And use cv2.VideoCapture("... a gstreamer pipeline ...") not cv2.VideoCapture(0)
At the first just grab and display image from camera and measure by putting hand to and fro that there is no delay. Play with the gstreamer until it is working fine. Then apply deep learning model but not at each frame just as you are able (but grab each frame in parallel, so you need two threads).