Hi! Having an issue where using cuda_FarnebackOpticalFlow is outputting None, i’ve tried a few different approaches and spent some time looking around for a solution.
Here’s a snippet of the code, any input would be greatly appreciated.
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
frame_next_gray = cv2.cvtColor(frame_next, cv2.COLOR_BGR2GRAY)
cframe_gray = cv2.cuda_GpuMat()
cframe_next_gray = cv2.cuda_GpuMat()
cframe_gray.upload(frame_gray)
cframe_next_gray.upload(frame_next_gray)
print(cframe_gray.size(), cframe_next_gray.size()) # Outputs the size correctly of the frames
cflow = cv2.cuda_GpuMat()
farneback = cv2.cuda_FarnebackOpticalFlow.create(5, 0.5, False, 21, 10, 7, 1.5, 0)
try:
farneback.calc(cframe_gray, cframe_next_gray, cflow)
flow = cflow.download()
except Exception as e:
print("ERROR: ", e)
if flow is not None: # flow is always None, for all frames i've tried.
print("Optical flow computed successfully for current frame.")
else:
print("Optical flow computation failed for current frame.")
Can you provide the output to print(cv2.getBuildInformation()) and frame and frame_next and I will take a look.
Here’s build information
General configuration for OpenCV 4.9.0 =====================================
Version control: 4.9.0
Extra modules:
Location (extra): C:/Users/-/Documents/GitHub/opencv-python/opencv_contrib/modules
Version control (extra): 4.9.0
Platform:
Timestamp: 2024-04-27T08:56:57Z
Host: Windows 10.0.19045 AMD64
CMake: 3.29.2
CMake generator: Visual Studio 17 2022
CMake build tool: C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
MSVC: 1939
Configuration: Debug Release
CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (16 files): + SSSE3 SSE4_1
SSE4_2 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (0 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (36 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (5 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
C/C++:
Built as dynamic libs?: NO
C++ standard: 11
C++ Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe (ver 19.39.33523.0)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /NODEFAULTLIB:atlthunk.lib /INCREMENTAL:NO /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:libcpmtd.lib /NODEFAULTLIB:msvcrtd.lib
Linker flags (Debug): /machine:x64 /NODEFAULTLIB:atlthunk.lib /debug /INCREMENTAL /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcpmt.lib /NODEFAULTLIB:msvcrt.lib
ccache: NO
Precompiled headers: YES
Extra dependencies: wsock32 comctl32 gdi32 ole32 setupapi ws2_32 cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/lib/x64
3rdparty dependencies: libprotobuf ade ittnotify libjpeg-turbo libwebp libpng libtiff libopenjp2 IlmImf zlib ippiw ippicv
OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: java world
Disabled by dependency: -
Unavailable: alphamat cannops cvv freetype hdf julia matlab ovis python2 sfm ts viz
Applications: -
Documentation: NO
Non-free algorithms: NO
Windows RT support: NO
GUI: WIN32UI
Win32 UI: YES
VTK support: NO
Media I/O:
ZLib: build (ver 1.3)
JPEG: build-libjpeg-turbo (ver 2.1.3-62)
SIMD Support Request: YES
SIMD Support: NO
WEBP: build (ver encoder: 0x020f)
PNG: build (ver 1.6.37)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.5.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: NO
DirectShow: YES
Media Foundation: YES
DXVA: YES
Parallel framework: Concurrency
Trace: YES (with Intel ITT)
Other third-party libraries:
Intel IPP: 2021.11.0 [2021.11.0]
at: C:/Users/-/Documents/GitHub/opencv-python/_skbuild/win-amd64-3.11/cmake-build/3rdparty/ippicv/ippicv_win/icv
Intel IPP IW: sources (2021.11.0)
at: C:/Users/-/Documents/GitHub/opencv-python/_skbuild/win-amd64-3.11/cmake-build/3rdparty/ippicv/ippicv_win/iw
Lapack: NO
Eigen: NO
Custom HAL: NO
Protobuf: build (3.19.1)
Flatbuffers: builtin/3rdparty (23.5.9)
NVIDIA CUDA: YES (ver 12.3, CUFFT CUBLAS)
NVIDIA GPU arch: 50 52 60 61 70 75 80 86 89 90
NVIDIA PTX archs: 90
cuDNN: NO
OpenCL: YES (NVD3D11)
Include path: C:/Users/-/Documents/GitHub/opencv-python/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python 3:
Interpreter: C:\Users\-\AppData\Local\Programs\Python\Python311\python.exe (ver 3.11.9)
Libraries: C:/Users/-/AppData/Local/Programs/Python/Python311/libs/python311.lib (ver 3.11.9)
numpy: C:/Users/-/AppData/Local/Temp/pip-build-env-99qmrikb/overlay/Lib/site-packages/numpy/core/include (ver 1.22.2)
install path: python/cv2/python-3
Python (for build): C:\Users\-\AppData\Local\Programs\Python\Python311\python.exe
Java:
ant: NO
Java: YES (ver 1.8.0.202)
JNI: C:/Program Files/Java/jdk1.8.0_202/include C:/Program Files/Java/jdk1.8.0_202/include/win32 C:/Program Files/Java/jdk1.8.0_202/include
Java wrappers: NO
Java tests: NO
Install to: C:/Users/-/documents/github/opencv-python/_skbuild/win-amd64-3.11/cmake-install
-----------------------------------------------------------------
Here’s the frames
Frame: [[[ 7 5 45]
[ 4 2 42]
[ 3 2 32]
...
[10 10 77]
[10 10 77]
[10 10 77]]
[[ 7 5 45]
[ 4 2 42]
[ 2 1 31]
...
[ 6 6 73]
[ 6 6 73]
[ 6 6 73]]
[[ 6 4 42]
[ 2 0 38]
[ 2 2 30]
...
[ 7 7 69]
[ 7 7 69]
[ 7 7 69]]
...
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 0 0 41]
[ 0 0 41]
[ 0 0 41]]
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 2 2 43]
[ 2 2 43]
[ 2 2 43]]
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 0 0 41]
[ 0 0 41]
[ 0 0 41]]]
Frame_next: [[[ 7 5 45]
[ 4 2 42]
[ 3 2 32]
...
[10 10 77]
[10 10 77]
[10 10 77]]
[[ 7 5 45]
[ 4 2 42]
[ 2 1 31]
...
[ 6 6 73]
[ 6 6 73]
[ 6 6 73]]
[[ 6 4 42]
[ 2 0 38]
[ 2 2 30]
...
[ 7 7 69]
[ 7 7 69]
[ 7 7 69]]
...
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 0 0 41]
[ 0 0 41]
[ 0 0 41]]
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 2 2 43]
[ 2 2 43]
[ 2 2 43]]
[[ 1 1 22]
[ 0 0 19]
[ 0 0 10]
...
[ 0 0 41]
[ 0 0 41]
[ 0 0 41]]]
I need the frames in a format I can use (e.g. .png, .jpg) to see what the issue is with your MRE.
Oh, sorry. Here’s the frames.
First image is frame 1 and second is frame 2
Hope this works, if there’s anything else i can provide let me know.
Its a problem in the way using the python bindings.
If you pass in an output argument when using the bindings you need to initialize it to the correct size and type.
e.g. If you want to pass in the output argument cflow which I recommend when using CUDA to avoid costly memory allocations. i.e.
farneback.calc(cframe_gray, cframe_next_gray, cflow)
you need to init cflow first
cflow = cv2.cuda_GpuMat(cframe_gray.size(),cv2.CV_32FC2)
The usual alternative which is how I always call functions from python to start with so I can determine the correct size and type is not to pass the output argument but it looks like the bindings here have a slight issue in that flow has to be an input arg
Help on method_descriptor:
calc(…)
calc(I0, I1, flow[, stream]) → flow
. @brief Calculates a dense optical flow.
.
. @param I0 first input image.
. @param I1 second input image of the same size and the same type as I0.
. @param flow computed flow image that has the same size as I0 and type CV_32FC2.
. @param stream Stream for the asynchronous version.
When this is the case you can still call the function without initializing flow by passing in an empty GpuMat and examinnig the resulting output GpuMat.
cflow = cv2.cuda_GpuMat()
cflow = farneback.calc(cframe_gray, cframe_next_gray, cflow)
Thank you! Spent the last two days trying to figure this out haha.
1 Like
No worries it confused me to start with as well!
