How to lower the delay in live stream of IP camera

Python
1

I’m doing a simple eye detection project on python. If I use the ip camera to record a video and implement the detection on it, there’s no delay. But if I use it to show the live stream and process the image there’s a few second’s delay and sometimes it crashes. When I use my laptop’s built-in webcam it works perfectly. The code I’m using is:

w = 960
h = 540
w2 = int(w/2)
h2 = int(h/2)

camera = cv2.VideoCapture('rtsp://169.254.189.11:554/user=admin_password=tlJwpbo6_channel=1_stream=0.sdp?real_stream')

#Haar Cascade
ap = argparse.ArgumentParser()
ap.add_argument("-c", "--cascade", required=True, help=r'C:\Users\ander\Desktop   \visao_computacional\olhos\haarcascade_eye.xml')
args = ap.parse_args(['-c', r'C:\Users\ander\Desktop\visao_computacional\olhos\haarcascade_eye.xml'])
detector = cv2.CascadeClassifier(args.cascade)

while True:
    (grabbed, frame) = camera.read()
    frame = cv2.resize(frame, (w, h))
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.GaussianBlur(gray, (5, 5), 0)
    gray = cv2.equalizeHist(gray)
    rects = detector.detectMultiScale(gray, minNeighbors=5, minSize=(90, 90), flags=cv2.CASCADE_SCALE_IMAGE)

    #Desenhar retângulo com classificação
    for (fX, fY, fW, fH) in rects:
        roi = gray[fY:fY + fH, fX:fX + fW]
        roi = cv2.cvtColor(roi, cv2.COLOR_GRAY2RGB)
        roi = cv2.resize(roi, (90,90))
        roi = roi.astype("float") / 255.0
        roi = img_to_array(roi)
        roi = np.expand_dims(roi, axis=0)
    
        classes = ["Superior Esquerdo", "Superior Direito", "Inferior Esquerdo", "Inferior Direito"]
        prev = olhos.predict(roi)  
        indice_prev_classe = np.argmax(prev, axis=1)[0]
        prev_classe = classes[indice_prev_classe]
        cv2.putText(frame, prev_classe, (fX, fY - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 255, 0), 2)
        cv2.rectangle(frame, (fX, fY), (fX + fW, fY + fH), (0, 255, 0), 2)
    
    cv2.imshow("Olhos", frame)

    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

camera.release()
cv2.destroyAllWindows()
3

What resolution is the IP Camera operating in? Try lowering it so you don’t have to resize and see what delay you have.

4

a pyrDown() might be faster than a generic resize()

also vary the parameters of the multiscale detection.

or maybe just run the code without any detection, and see the latency there.

5

Crashing is, obviously, what happens when read() fails, which it may do even when things are ok, and your code pretends after the failed call that the imaginary frame is good to work on