Optical Flow on Array of Points

Python
1

Hi All,

I’m working on a project that tracks objects moving in front of a camera, for instance to estimate their speed. I am using Lucas-Kanade sparse optical flow (from the OpenCV samples) to track the movement which works well.

The objects can also be separated nicely using the SimpleBlobDetector, providing an array of centroids/keypoints for each frame.

I’m wondering what the best* approach might be to perform frame-to-frame tracking on the keypoint array generated by SimpleBlobDetector rather than on the image frame itself? I know that LK can be initialized with keypoints but also requires an image input for each frame. I would like to completely eliminate the use of image inputs and only use arrays of a few keypoints for tracking.

I’ve spent a fair amount of time searching this but haven’t found any good answers. Any pointers are highly appreciated.

*Best for this application would be fast and as robust as possible

Thanks,
West

2

LK is optical tracking.

if you have a bunch of keypoints, there’s nothing to see anymore.

you’re asking for tracking by association, i.e. associating detections (over time) into tracks. LK doesn’t do that at all.

you can use blob detection to get positions that begin your tracks, and then generate tracks with LK given those positions.

3

Hi @crackwitz,

That totally makes sense. Let me rephrase my question a little: Is there an approach that yields a similar output like LK optical flow but operates on keypoints only? Of course I can take blob detections from successive frames and create tracks manually, but that doesn’t check whether there’s a plausible connection between those detections from frame to frame if multiple blobs are detected. I’m wondering if there’s an algorithm that e.g. estimates a motion vector for each point and then finds matching points in the next frame or implements some similar validation approach?

I’m thinking of this like motion capture approaches used in film, where not the image of the person moving is tracked but rather a few very high-contrast points.

Best,
West