calcOpticalFlowSparseRLOF Assertion Error in InitialFlow

Hello, I am trying to use SparseRLOF Optical Flow to calculate displacement of falling particles. I have already done this using Lucas Kanade and it worked fine. I wanted to give this algorithm a shot and see if it does better according to some metrics. Unfortunately, I a facing an error when I call the calcOpticalFlowSparseRLOF() function. I have pasted the error below. I have also pasted my code below. Please let me know if you can help. This is my first time using this forum to post. If I need to reformat, please let me know.

Code:

vector<cv::Point2f> last_point, current_point;

        last_point.reserve(last_keypoint.size());
        current_point.reserve(last_keypoint.size());
        float ferr = 30;
        float fmax_fbklt_dist = 2;
        vector<uchar> vstatus; // 特征点跟踪成功标志位
        vector<float> verr;    // 跟踪时区域误差和

        KeyPointsToPoints(last_keypoint, last_point);

        // KeyPointsToPoints(current_keypoint, current_point);

        cv::calcOpticalFlowPyrLK(last_img, current_img, last_point, current_point, vstatus, verr, winsize, maxlevel);
        auto &vpriorkps = current_point;
        auto &vkps = last_point;
        std::vector<cv::Point2f> vnewkps;
        std::vector<cv::Point2f> vbackkps;
        std::vector<int> vkpsidx;
        size_t nbkps = last_keypoint.size();
        vnewkps.reserve(nbkps);
        vbackkps.reserve(nbkps);
        vkpsidx.reserve(nbkps);
        vkpstatus.reserve(nbkps);
        size_t nbgood = 0;

        for (size_t i = 0; i < nbkps; i++)
        {
            if (!vstatus.at(i))
            {
                vkpstatus.push_back(false);
                continue;
            }

            if (verr.at(i) > ferr)
            {
                vkpstatus.push_back(false);
                continue;
            }

            // if (!inBorder(vpriorkps.at(i), vcurpyr.at(0)))
            // {
            //     vkpstatus.push_back(false);
            //     continue;
            // }

            vnewkps.push_back(vpriorkps.at(i));
            vbackkps.push_back(vkps.at(i));
            vkpstatus.push_back(true);
            vkpsidx.push_back(i);
            nbgood++;
        }

        if (vnewkps.empty())
        {
            return;
        }

        // vstatus.clear();
        // verr.clear();
        // cout << "可追踪的点数1:" << nbgood << endl;

        // cv::calcOpticalFlowPyrLK(current_img, last_img, vnewkps, vbackkps,
        //                          vstatus, verr, winsize, 0);
        // nbgood = 0;
        // for (int i = 0, iend = vnewkps.size(); i < iend; i++)
        // {
        //     int idx = vkpsidx.at(i);

        //     if (!vstatus.at(i))
        //     {
        //         vkpstatus.at(idx) = false;
        //         continue;
        //     }

        //     // if (cv::norm(vkps.at(idx) - vbackkps.at(i)) > fmax_fbklt_dist)
        //     // {
        //     //     vkpstatus.at(idx) = false;
        //     //     continue;
        //     // }
        //     nbgood++;
        // }
        vector<cv::Point2f> vnew_new=vbackkps;
        if(vbackkps.size()==vnew_new.size())
            cout<<"xxx"<<endl;
        // for (int i = 0; i < vbackkps.size(); i++)
        // {
        //     vbackkps_new.push_back(cv::Point2f(vbackkps[i].x, vbackkps[i].y));
        // }
        cout<<sizeof(vnew_new[0].x);
        cout << "可追踪的点数2:" << nbgood << endl;
        cout << "---objtracking function:" << last_point.size() << endl;
        vstatus.clear();
        verr.clear();
        auto test_point=last_point;
        cv::Ptr<cv::optflow::RLOFOpticalFlowParameter> rlof_ptr = cv::optflow::RLOFOpticalFlowParameter::create();
        rlof_ptr.get()->setUseInitialFlow(true);
        cv::optflow::calcOpticalFlowSparseRLOF(last_color_image, cur_color_image, last_point, test_point, vstatus, verr, rlof_ptr);

Error

what():  OpenCV(4.5.5) /home/docker_file/opencv_contrib-4.5.5/modules/optflow/src/rlof/rlof_localflow.cpp:402: error: (-215:Assertion failed) nextPtsMat.checkVector(2, CV_32F, true) == npoints in function 'calcLocalOpticalFlowCore'

I used the same exact vector formats for the Lucas Kanade method and am therefore confused as to why this problem is occurring with SparseRLOF. Any input would be appreciated.

This post has a similar problem with me:

(https://forum.opencv.org/t/cv2-optflow-calcopticalflowsparserlof-giving-assertion-error/4835)

I use sizeof to determine the type and return 4, which proves float32.

In the code, I tried to use prepts replication so that the data format would be exactly the same, using UseInitialFlow, but still reported the same error. Is there a bug in the source code? Initialoptflow cannot be opened

1 Like

I’ve observed the same problem :). I have created a PR, feel free to try:

1 Like