C++ opencv_ffmpeg 3.3.1 adding metadata to video container

1

Hi all,

I have build & installed 3.3.1 on my machine without problems. I have also been able to create a h264 encoded file with the precompiled opencv_ffmpeg binaries that is download during building. I can save the file a .mp4 container but it gives me an error

OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'

but the file is created. I can also save the file in an .avi container without any errors but not sure if this is the right choice of container for me, because I want to add some metadata for each frame that I add to the movie file/container. I use opencv’s videowriter to write the files. So how can I get some metadata in a video container using the Opencv Video I/O API

2

I’d recommend against using .avi, especially with “modern” codecs.

try .mkv if you don’t like MP4. an MPEG transport stream (.mts) is a good idea in some situations.

ISO MP4 is a somewhat strictly defined container that doesn’t allow arbitrary codecs. it’s based on QuickTime MOV.

try asking for “avc1” or “AVC1” instead of “H264”. same thing but perhaps that makes a difference.

3

I concur. I used mkv with h264 encode and increased my video frame rate 5X and reduced the resulting file size compared to AVI with bmp pixel format by 10x. It was not lossless, but it was “visually lossless” and good enough for further image analysis.

4

thank you guys, I have tested it with avc1 with the mp4 file and it works no errors of any sort. But I think I’ll take your advice and use the .mkv format. Is it possible to add the metadata ( like frame number, GPS number… ) to this container using the OpenCV Video I/O API (videowriter )?

5

nope. OpenCV VideoWriter is bare. if you need flexibility, use ffmpeg or other APIs directly…

a simple solution: write first, then remux using ffmpeg CLI and add in the metadata you need. GPS stands for global positioning system, i.e. coordinates on the planet. nobody ever calls those coordinates “a GPS number”

“frame numbers” aren’t metadata though. they’re inherent in the idea of a video file.

6

Sorry my bad, I meant GPS coordinates. The frame number that I’m referring to is one that comes from the camera module which might not be 0 at the moment I start creating/recording the movie file. When you say “use …directly” , it means that there is no way that it can’t be done within OpenCV. Or can I use something else as a backend instead of ffmpeg? Actually I was hoping to do this with OpenCV Video I/O or so…

7

OpenCV’s videoio gives you everything that is documented. if it’s not documented, it probably doesn’t exist.

OpenCV’s videoio is not intended for what you’re planning to do and it can’t do it either.

if your camera gives you frame numbers or timestamps… you should want to learn about “presentation timestamps”.

8

crackwitz, thank you for your advice. It seems that I have been barking up to the wrong tree since videoio isn’t able to do this. I guess I will wrap ffmpeg.exe in a dll and link against it. Then I will be able to use ffmpeg directly. Any idea how I can do this in memory if I have a cv::Mat and the corresponding metadata ? any hints or links would be great

9

you don’t wrap ffmpeg.exe actually, but use the dlls of its libraries

https://trac.ffmpeg.org/wiki/Using%20libav*

if your task is simple enough, you might get away with running the mentioned postprocessing as a shell command (subprocess). generate the file using OpenCV’s VideoWriter, then use ffmpeg to mux the metadata and the first file together into a second file.

you need to know what you have to do for your metadata to go in there. explaining ffmpeg’s APIs is beyond the scope of this forum. they have their own specialized support… and documentation including samples.