cv2.optflow.calcOpticalFlowSparseRLOF Giving Assertion Error

Python
1

Hello, I am trying to use SparseRLOF Optical Flow to calculate displacement of falling particles. I have already done this using Lucas Kanade and it worked fine. I wanted to give this algorithm a shot and see if it does better according to some metrics. Unfortunately, I a facing an error when I call the calcOpticalFlowSparseRLOF() function. I have pasted the error below. I have also pasted my code below. Please let me know if you can help. This is my first time using this forum to post. If I need to reformat, please let me know.

The code:

#importing the necessary libraries
import cv2
import numpy as np
import matplotlib.pyplot as plt
import math

a = 9.81*1000*2.8


y0 = 140 #v0 = 0 at y0
dy = np.arange(0, 1081-y0, 1)
y_vals = np.arange(y0,1081,1)
v = np.sqrt(2*a*(dy))
zip_iterator = zip(y_vals, v)
velocity_dict = dict(zip_iterator)

fps = 240

# Creating a VideoCapture object to read the video
cap = cv2.VideoCapture('quinoa_short.mp4')
orb = cv2.ORB_create(nfeatures =10000)
ret, old_frame = cap.read()

# Loop untill the end of the video
loop_var = 1
normalized_sum = 0
while (cap.isOpened()):
    # Capture frame-by-frame
    ret, new_frame = cap.read()
    print(loop_var)
    if new_frame is None:
    	break
    kp, des = orb.detectAndCompute(new_frame, None)
    p0 = []
    p_guess = []
    for i in kp:
        #filter points
    	if i.pt[1] >=225 and i.pt[0] <= 1190:
            p0.append(i.pt)
            x = i.pt[0]
            y = i.pt[1] + velocity_dict[math.floor(i.pt[1])]/fps + (a/2)*(1/fps)**2
            point = (x, y)
            p_guess.append(point)
    #optical flow 
    retval	=	cv2.optflow.RLOFOpticalFlowParameter_create()
    retval.setUseInitialFlow(True)
    p1, st, err = cv2.optflow.calcOpticalFlowSparseRLOF(old_frame, new_frame, np.float32(p0), nextPts = np.float32(p_guess), rlofParam = retval)

The error:

Traceback (most recent call last):
  File "/Users/vishnu/Documents/Summer Project/Metrics Folder/RLOF/Quinoa/gravity_sim.py", line 58, in <module>
    p1, st, err = cv2.optflow.calcOpticalFlowSparseRLOF(old_frame, new_frame, np.float32(p0), nextPts = np.float32(p_guess), rlofParam = retval)
cv2.error: OpenCV(4.5.1) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-p3unqtkr/opencv_contrib/modules/optflow/src/rlof/rlof_localflow.cpp:402: error: (-215:Assertion failed) nextPtsMat.checkVector(2, CV_32F, true) == npoints in function 'calcLocalOpticalFlowCore'

I used the same exact vector formats for the Lucas Kanade method and am therefore confused as to why this problem is occurring with SparseRLOF. Any input would be appreciated.

2

try some wild reshaping, like to (-1, 1, 2)

3

I tried that. It did not work. Is the function working properly? I’m not sure if it is something I’m doing or if there is something wrong with the function itself.

4

convert p_guess to numpy array

post these:
p_guess.shape
p_guess.dtype
p_guess.data.contiguous

and show how you tried to set shape = (-1, 1, 2)

crosspost:

5

Hello,
My reshape line is shown below:
p_guess = np.array(p_guess).reshape(-1,1,2)

printing p_guess.shape returns:
(5589, 2)
printing p_guess.data.contiguous returns:
True
printing p_guess.dtype returns:
float64
These outputs are without reshaping. reshaping just changes the shape (5589, 1, 2) where 5589 is the number of key points.

6

it needs to be float32, so ask for .astype(np.float32) or see about causing that type to begin with

7

Right, I resolved that error when doing the Lucas Kanade algorithm. In this line, I pass in p_guess by converting it to float32:
p1, st, err = cv2.optflow.calcOpticalFlowSparseRLOF(old_frame, new_frame, np.float32(p0), nextPts = np.float32(p_guess), rlofParam = retval)