does set(cv2.CAP_PROP_FORMAT, -1) work with grab and retrieve? as with read? because i need to use grab to synchronize two mjpeg webcams.
i used video_capture_1 = cv2.VideoCapture(1, cv2.CAP_DSHOW).
Yes but it only works with the ffmpeg backend.
it works with the ffmpeg backend even if the source is a webcam?
can you also make it for Directshow?
I think on win10 webcam was drived by directshow by default. ffmpeg has to use directhow input (i might be wrong).
You have to use CAP_FFMPEG but I don’t know if you can access the webcam using ffmpeg as described here from VideoCapture and even if you can I don’t know what format directshow would output to ffmpeg. It may only output the decoded frame. Sorry I can’t be of more help, it’s an interesting question and I’ll take a look when I have a moment.
Why do you want the raw encoded data?
ok. we are doing VR recording and streaming. for human eyes to feel comfortable. it has to be high resolution because of the big field of view. Two 2k and 4k cameras are obsolute. i have tested two 2K cameras. because of the decoding in the Retrieve, the cpu usage always went up to 60%. and fps was reduced.
it is not a commerical project. but VR online meeting, chatting and streaming are becoming popular. so people do need it. i can not imagine people using bulky 3000 us dollars VR camera for online VR chatting in their home. they should simply use two usb webcam to do it. i really hope you can find time to do it. bests.
I performed a quick test and it appears that you can retrieve the raw encoded MJPEG data from directshow using CAP_FFMPEG if you slightly modify the source and link OpenCV against libavdevice. You need to add
avdevice_register_all();
to void CvCapture_FFMPEG::init() and
#include <libavdevice/avdevice.h>.
Unfortunatley this means you need to build OpenCV against FFMPEG libs and not use the precompiled dll - opencv_videoio_ffmpegxxx_64.dll.
Once the modification is in place you simply need to add the following environmental variable
OPENCV_FFMPEG_CAPTURE_OPTIONS=input_format;dshow
and open as
VideoCapture cap(“video=<NAME_FROM_DEVICE_MANAGER>”);
cap.set(CAP_PROP_FORMAT, -1);
to retrieve the raw data, or pass it straight through to cuda for decoding by opening as
cv::Ptr<cv::cudacodec::VideoReader> reader = cv::cudacodec::createVideoReader(String(<NAME_FROM_DEVICE_MANAGER>));
where <NAME_FROM_DEVICE_MANAGER> is the name of your webcam in device manager, in my case this would be
VideoCapture cap(“video=HD Webcam”);
I can’t guarantee that these modifications won’t break OpenCV videoio somewhere so proceed with caution. I also cannot guarantee that all the info you need to decoded the retrieved frame will be present (see below), although I can’t think of a reason why not.
I am not familiar with the format of MJPEG but the following is the start of the hex from a raw frame, does it look correct?
ff d8 ff c0 00 11 08 01 e0 02 80 03 01 21 00 02 11 01 03 11 01 ff db 00 84 00 08 05 05 08 05 05 08 08 08 08 0b 08 08 0b 0e 17 0e 0e 0b 0b 0e 1d 14 14 11 17 22 1d 22 22 1f 1d 1f 1f 26
28 34 2e 26 28 31 28 1f 1f 2e 40 2e 31 37 3a 3c 3c 3c 22 2b 42 45 40 3a 45 34 3a 3c 3a 01 08 0b 0b 0e 0b 0e 1a 0e 0e 1a 3a 26 1f 26 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a
3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a 3a ff c4 01 a2 00 00 01 05 01 01 01 01 01 01 00 00 00 00 00 00 00 00 01 02 03 04 05 06
07 08 09 0a 0b 01 00 03 01 01 01 01 01 01 01 01 01 00 00 00 00 00 00 01 02 03 04 05 06 07 08 09 0a 0b 10 00 02 01 03 03 02 04 03 05 05 04 04 00 00 01 7d 01 02 03 00 04 11 05 12 21 31
41 06 13 51 61 07 22 71 14 32 81 91 a1 08 23 42 b1 c1 15 52 d1 f0 24 33 62 72 82 09 0a 16 17 18 19 1a 25 26 27 28 29 2a 34 35 36 37 38 39 3a 43 44 45 46 47 48 49 4a 53 54 55 56 57 58
1 Like