I am trying to save some footage from a webcam but am coming across some issues. My webcam supports compressed (MJPG) and raw (YUYv 4:2:2) formats and I am trying to get MJPG due to the faster fps. I have simplified my code (below) and have reproduced the issue.
import cv2
from datetime import datetime, timedelta
fourcc = cv2.VideoWriter_fourcc(*'avc1')
fn = "out.mp4"
cap = cv2.VideoCapture(0, cv2.CAP_V4L2) #tried with and without
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"MJPG")) #also tried with and without
fps = int(cap.get(cv2.CAP_PROP_FPS))
out = cv2.VideoWriter(fn, fourcc, fps, (int(cap.get(3)), int(cap.get(4))))
frames_written = 0
print(f"FPS {fps}")
show = False
end = datetime.utcnow() + timedelta(seconds=20)
while end >= datetime.utcnow():
ret, frame = cap.read()
if not ret:
raise Exception("NO FRAME")
out.write(frame)
frames_written+=1
if show:
cv2.imshow("f", frame)
cv2.waitKey(33)
cap.release()
out.release()
cv2.destroyAllWindows()
When I try to use the webcam video device directly in OpenCV, the recorded footage although smooth, is sped up.
I have also tried creating a dummy video device and sending ffmpeg output to it using the following command:
When I try to use the dummy video device as the src the first couple seconds seem normal speed but is choppy but then speeds up and then is less choppy. Any help would be appreciated. I have been banging my head for the last couple of days.
you can’t expect the frame rates of those things to be constant or consistent. the “smarter” it is, the more it’ll mess with its own frame rate.
besides, “frame rate” is an illusion. work with “presentation timestamps”, both for capturing and for storing the video. you’ll get those PTSes if you use ffmpeg or V4L2 API directly.
When I use it in VLC the recorded footage is 1920x1080 with 30fps resolution. Although I am new to OpenCV I think the issue is it’s not using the compressed stream but still expecting 30fps instead of the 10fps (due to it being YUYV format) so the video is sped up. How do I force getting MJPG instead of raw footage from the webcam. It shouldn’t be this difficult.
EDIT:
It’s worth mentioning that the VLC recorded video file size for a 10 second video is 1.3gb which means its storing it in its uncompressed format (although there is no lag or choppiness).
I thought the following ffmpeg command would pipe the compressed version to the video capture:
getting it to request mjpeg requires the props set in the right order. the fourcc either needs setting first or last. then it’ll work.
if it doesn’t, you will see a slower frame rate.
nothing should be “sped up”. if you query the frame rate, it’ll be the effective nominal rate, i.e. the device will aim to give you frames at that rate (within limits of physical perfection).
there’s too much going on here. you’re involving some virtual video device for some tests. you’re testing both the reading and writing in concert. that won’t tell you where the issue lies.
remember, video compression costs. if it costs too much, it won’t be able to keep up with the producer (camera). if you wanted to store the mjpeg coming from the camera, without recompression, use ffmpeg. it’s probably capable of that. OpenCV might have a way to get that done, but probably not, and that (copying compressed streams) isn’t even an intended use case for OpenCV.
There is a lot going on here because I have tried everything people have recommended.
I have tried setting mjpeg first, last and middle. The reading by itself is fine. It when there needs writing to be done.
My use case in my full program is using OpenCV to search for certain triggers in a frame and when it has been found, record until another certain condition is met. I was doing a bulk of the testing with prerecorded footage for practicality sake and thought the transition to a webcam would be easier (my mistake).
Sorry I am new to this and may be outside of the scope of this forum but do the webcams give compressed data natively without having to use your pc’s processing power? If so how would this be done and would you have to set it in OpenCV if you use ffmpeg or somehow force webcam to only output mjpg? I know my ffmpeg is not correctly working because when I record using VLC to the virtual device, a 10 second video is more than 1 gbs.
yep. they do that to stay within the data rate budget of the USB (USB 2) hub. MJPEG is the easiest and oldest compressed mode in USB Video Class (UVC) devices. basically every webcam above 640x480 offers such modes. uncompressed 1080p would require in excess of one gigabit/sec.
that’s also the reason why starting a camera might sometimes take a minute, if you have two or more cameras on the same hub. the involved drivers have to negotiate some modi to squeeze the second video into what’s left after the first camera got its fill. that’ll make the picture be low resolution, low frame rate, have terrible compression artefacts, …
VLC, same as OpenCV, will decompress the video, in the expectation that someone wants the pictures, not the compressed stream. you can tell VLC to recompress. for storage, ffmpeg always applies some (intentionally bad) default compression. from what you say, VLC probably doesn’t.
you can tell ffmpeg to use the “copy” codec. then you can direct the mjpeg stream coming from the webcam to file or wherever, and no decompression or recompression is taking place.
in your code, you should be fine just recompressing (with VideoWriter) whatever frames you want to keep. you tried the avc1 codec for storage. maybe try MJPG instead. H.264 costs more than MJPEG. if the compression cost was the issue, switching to MJPEG for the output should immediately show an improvement. CPU budget is more of an issue if you happened to run this on a raspberry pi.
anyway, I can’t actually tell the reason for your frame rate issues. I only have enough information to guess and make hypotheses.