VideoCapture Silence Warning - [h264 @ 0x559351d7b040] mmco: unref short failure

Python
1

I’m using VideoCapture to load a video’s frames in batches. Eventually, my terminal starts spamming me with the following error, despite the program will running:
[h264 @ 0x559351d7b040] mmco: unref short failure
[h264 @ 0x559351d7b040] mmco: unref short failure
[h264 @ 0x559351d7b040] mmco: unref short failure
[h264 @ 0x559351d7b040] mmco: unref short failure
...

What’s causing this?
Is there any way to stop all warnings messages?

Sample Code:

vid = cv2.VideoCapture(vid_path)

    # Prepare `num_processors` batches of images
    for frame_num in range(start_frame, start_frame + batch_size):
                         
        vid.set(cv2.CAP_PROP_POS_FRAMES, frame_num)
        ret, frame = vid.read()
        if ret:
            frame = Image.fromarray(frame)
            frame = frame_transforms(frame)
            self.batch.append(frame)
2

run this on your video file:

ffmpeg -i yourfile.something -f null -

it will analyze the file.

3

Here’s the output:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '0015, 2017, England Open, QF, Lin Dan (CHN) vs Viktor Axelsen (DEN).mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
  Duration: 01:23:40.11, start: 0.000000, bitrate: 1063 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 931 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
    Stream #0:0(und): Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.54.100 wrapped_avframe
    Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.54.100 pcm_s16le
frame=125501 fps=1535 q=-0.0 Lsize=N/A time=01:23:40.10 bitrate=N/A speed=61.4x    
video:65692kB audio:864792kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
4

ah, you are seeking. OpenCV doesn’t do that too well. I would suggest reading frames in order, without seeking… or seeking once, since you have consecutive frames anyway.

1 Like
5

Do you mean something like this?

I moved vid.set to occur only once - outside the for loop.

vid = cv2.VideoCapture(vid_path)
    
        # CHANGE 
        vid.set(cv2.CAP_PROP_POS_FRAMES, frame_num)

        for frame_num in range(start_frame, start_frame + batch_size):
            
            ret, frame = vid.read()
            if ret:
                frame = Image.fromarray(frame)
                frame = frame_transforms(frame)
                self.batch.append(frame)
6

that’s definitely better.

seeking in videos is difficult and OpenCV is not a media API. files might index their video data by time stamp, or they might not. I don’t know if common container formats support indexing by frame number.

if you need to be absolutely sure, either don’t seek and vid.grab() until you’re at the right frame (possibly cheaper than read = grab + retrieve), or work with something like PyAV. there you have full control.

2 Likes
7

Sorry for the delayed reply, and thank you so much for the feedback!
I’ll try it out and see how it goes.

8

oh btw, common video codecs these days work with keyframes (I-frames) and following “predicted” frames referencing the previous keyframe. you can only seek to frames that are I-frames. if you try to seek to anywhere else, you’ll likely end up on the nearest I-frame after the time you wanted. it’s all very complicated and impossible to explain in a paragraph.

1 Like
9

Hi, for one of my algorithms, I need to get previous frames in a video - how can I do this without continuously using vid.set?
(I did do some research - it seems there isn’t an obvious way)

10

how many? if it’s a limited number at the same time, keep them around yourself.

one: use a variable
multiple, arbitrary index: hashmap, index → frame
multiple in sequence: a queue or list or vector or something

11

Well, an obvious way is to store the read frames in a circular buffer/list of suitable size, and when you have a hit, look up back into it. Obviously it takes some memory to store the frames.

12

@crackwitz @matti.vuori
I need to get frames backwards in a video, and compare each (one at a time) with a pre-set frame, and will stop when some condition is met (this happens anywhere from ~1000-5000 times).
I’m currently just using vid.set continuously, and saving the frame received in a variable. This works, but I want to prevent the [h264 @ 0x559351d7b040] mmco: unref short failure warning message.