Bug: Brox Optical Flow (CUDA) with opencv-python giving errors

Python
,
1

I have been trying to use the CUDA implementation of Brox Optical Flow, but no Python documentation is available. I managed to use create(), but I keep running into errors while using the calc() function.

I’m using a syntax similar to the example code I found for the Nvidia optflow implementation in CUDA on the GitHub repository, but I’m getting the following errors.

Type current =  5
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-8-3dce7b98f55d> in <module>
     61     gpu_frame = cv2.cuda.cvtColor(gpu_frame, cv2.COLOR_BGR2GRAY)
     62     print("Type current = ", gpu_frame.type())
---> 63     flow = brox_of.calc(gpu_prev, gpu_frame, None)
     64 
     65     gpu_prev = gpu_frame

error: OpenCV(4.5.2) C:\OpenCV_Build\opencv_contrib-4.5.2\modules\cudaoptflow\src\brox.cpp:134: error: (-215:Assertion failed) frame0.type() == CV_32FC1 in function '`anonymous-namespace'::BroxOpticalFlowImpl::calc'

I initially realised that my inputs were RGB 8U and changed them to CV_32FC1 (verified with type() function, got the output as 5), but I’m still getting the same error.

This is the definition of “brox_of” if anyone thinks its related to the error.
brox_of = cv2.cuda_BroxOpticalFlow.create(0.197, 50.0, 0.8, 5, 150, 10)
This line did not give me any errors.

TLDR;
I can’t figure out how to use the calc() function to calculate Brox optical flow in python. The error says input isn’t 32FC1, but I have verified that it is.

import cv2
import numpy as np
print(cv2.cuda.getCudaEnabledDeviceCount())
vidPath = 'C:/Users/Aditya/Downloads/JESTER DATASET/20bn-jester-v1/991/%5d.jpg'
lr = 0.05
check_res = False
frame_device = cv2.cuda_GpuMat()

#taking input, pass 0 to use webcam data
cap = cv2.VideoCapture(vidPath)

bgmog2_device = cv2.cuda.createBackgroundSubtractorMOG2()
brox_of = cv2.cuda_BroxOpticalFlow.create(0.197, 50.0, 0.8, 5, 150, 10)

def bgmogCuda(frame,lr,store_res=False):
    frame_device.upload(frame)
    fg_device = bgmog2_device.apply(frame_device,lr,cv2.cuda.Stream_Null())
    fg_host = fg_device.download()
    if(store_res):
        gpu_res.append(np.copy(fg_host))
    return fg_host
        
gpu_res = []

gpu_prev = cv2.cuda_GpuMat()
gpu_frame = cv2.cuda_GpuMat()

ret, frame = cap.read()
frame = np.float32(frame)/255.0
#prev_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gpu_prev.upload(frame)
gpu_prev = cv2.cuda.cvtColor(gpu_prev, cv2.COLOR_BGR2GRAY)
print("Type prev = ",gpu_prev.type())

h, w = frame.shape[:2]
temp = np.zeros((h, w), np.float32)
#temp = gpu_prev.download()
#print(cv2.cuda_GpuMat.type(gpu_prev))
cv2.imshow('temp', temp)

#loop frames
while(1):

    #for i in range(3):
    ret, frame = cap.read()
    
    if not ret:
        cap.release()
        break
     
    #this block is for background subtraction
    fgmask = bgmogCuda(frame, lr, store_res=check_res)
    kernel = np.ones((3, 3), np.uint8)
    erode = cv2.erode(fgmask, kernel, iterations = 1)   
    cv2.imshow('bgmog', erode)
    
    #brox optical flow opencv cuda

    frame = np.float32(frame)/255.0
    gpu_frame.upload(frame)
    gpu_frame = cv2.cuda.cvtColor(gpu_frame, cv2.COLOR_BGR2GRAY)
    print("Type current = ", gpu_frame.type())
    flow = brox_of.calc(gpu_prev, gpu_frame, None)
    
    gpu_prev = gpu_frame
    
    if 0xFF & cv2.waitKey(50) == 27:
        break
        
cv2.waitKey(0)
cv2.destroyAllWindows()

Edit: After some debugging, I’ve found that the type of gpu_prev changes when gpu_frame is updated. Seems like a bug to me

2

please, no images of code or errors here, you make it unnessecary hard for ppl to help you, (replace with TEXT, thanks).

1 Like
3

I’ll update it right away. Thanks for letting me know!
Edit: Done!

1 Like
4

I’ve updated it for easier readability or if anyone wants to copy paste the error.