I am developing on the RK3588 platform. The signal from the HDMI IN interface comes through the device node /dev/video40. When the input signal is in RGB format, cv2.VideoCapture("/dev/video40") works perfectly.
However, if the input signal is YUV444 or YUV422, cap.isOpened() returns False. and cap.read() returns ret=False.
When I try cv2.VideoCapture("/dev/video40", cv2.CAP_V4L2), the following warning pops up:
[ WARN:0@317.790] global cap.cpp:215 open VIDEOIO(V4L2): backend is generally available but can’t be used to capture by name
Using the device index also fails. The command cv2.VideoCapture(40, cv2.CAP_V4L2) outputs these warnings:
[ WARN:0@355.408] global cap_v4l.cpp:913 open VIDEOIO(V4L2:/dev/video40): can’t open camera by index
[ WARN:0@355.408] global cap.cpp:478 open VIDEOIO(V4L2): backend is generally available but can’t be used to capture by index
How can I read this YUV data properly?
You can try writing a gstreamer pipeline and opening it as a string to VideoCapture
That is the common way to video on Jetson devices
You can also browse OpenCV’s issues (open and closed ones). Perhaps there is some wisdom there. If not, you could file an issue. I would expect V4L2 to work, or at least to have better warnings and errors than what OpenCV gave you.
Thank you for your reply. The OpenCV included with my firmware does not have the GSTREAMER backend. I tried asking AI for help and compiled and installed it myself, then attempted to read the stream using the pipeline:
gst_pipeline = "v4l2src device=/dev/video40 ! video/x-raw,fps=60 ! videoconvert ! appsink".
However, the frame rate was extremely low, and none of the various solutions I found through AI worked.
I then tried using FFmpeg to read the pipeline and fetch the data via Python. Unfortunately, I encountered an issue where my frame consumption cannot keep up with FFMpeg’s reading speed. The input is 60 FPS, but my processing speed is only 30–40 FPS (with all other code identical, this is even lower than the 50+ FPS I get when reading RGB frames with cv2). This causes a frame backlog, making the video slow, laggy, and stuck on footage from several seconds ago — with the delay getting worse over time.
I will also look for solutions on GitHub. Thanks again for your help!
Although cv2 also doesn’t reach 60 FPS, cap.read() always returns real-time frames for me and does not cause the same kind of frame backlog issues as FFmpeg. So I really hope to resolve this issue within OpenCV itself.
When receiving RGB input, I tested VideoCapture(const String &filename, int apiPreference=CAP_ANY) with different apiPreference values including cv2.CAP_FFMPEG and cv2.CAP_V4L2. Only cv2.CAP_V4L2 worked for frame capture. Therefore I suspect OpenCV automatically falls back to the V4L2 backend when no apiPreference is specified. Using cv2.CAP_FFMPEG throws an error consistent with my earlier log:
VIDEOIO(FFMPEG): backend is generally available but can’t be used to capture by name.
However, none of these backends work when the input switches to YUV format.
I also tried parameter configuration via cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('Y', 'U', 'Y', 'V')) and similar calls, all returning False, and cap.isOpened() stays False permanently.
BTW, Is it feasible to leverage the FFmpeg backend and read video via a pipeline string syntax similar to the earlier gst_pipeline definition?
you could try linuxpy · PyPI
you should file an issue on OpenCV’s github so someone can look into this or help you look into this
I would strongly advise AGAINST sending video data between processes through pipes. it’s costly, compared to using a library.
you could use PyAV (av · PyPI), because that is ffmpeg as a library.