there is a difference between dense and sparse flow. sparse flow is calculated for a given set of points only, not all pixels. theoretically, I think LK can be formulated for dense flow. AFAIK OpenCV has LK for sparse flow only.
besides, why stick to 20 and 40 year old algorithms?
use “DIS” optical flow. it’s fast and has lots of other positive attributes.
you can swap out flow algorithms, if that’s not too much hassle. however, don’t speculate so much on what might be wrong with its performance until you’ve profiled the program. then you know where time is spent and where optimization is worth it.
CUDA is far from the only way to use a GPU. OpenCV uses OpenCL in many of its basic functions, if you give it UMat types instead of Mat types.