I am experimenting OpenCV’s integration with CUDA (cv::cudacodec) using the official example here. The sample code works. However, it seems to me that the resultant file generated by cv::Ptr<cv::cudacodec::VideoWriter> is an H264 file, without a proper container such as mp4/avi.
My question is whether or not there is a recommended way to wrap the h264 file (say in-memory?) into a proper mp4/avi file.
I can think of one circuitous approach to achieve this–apart from building OpenCV with cv::cudacodec, we firstly build FFmpeg with Nvidia Video codec and then build OpenCV with this particular FFmpeg build, then we use cv::VideoWriter’s FFmpeg backend to leverage hardware codec. But wondering if there is a more straightforward way…
When I added cudacodec::VideoWriter back in I didn’t include container formats as I wasn’t sure if it was really necessary and I didn’t want to have to mess with cv::VideoReader to get to FFMpeg on the backend.
If your requirement for a container format is that strong then that would be a reasonable approach. Last time I checked writing a 1080p h264 file with cv::cudacodec::VideoWriter was neary twice (540fps vs 260fps) as fast as cv::VideoWriter with hardware acceleration.
The main use cases for cv::cudacodec::VideoWriter would be
when your data is already on the device, maybe your reading video with cv::cudacodec::VideoReader,
when you require maximum encoding performance (not that common, 540 fps is probably not that important to most people when the rest of their pipeline can only processing images at say 20 fps),
when you want to lower the processing overhead on the CPU (cv::VideoWriter with hardware accel still stresses the CPU).
I tested a few players.
Windows Media Player and Windows 11’s “Photos” can’t play .h264 file.
Also, VLC can only play the file if the extension if “.h264”. If I set the extension to “mp4”, VLC doesn’t work either.
This is what I can achieve, but still figuring if I can avoid using the FFmpeg backend altogether–invoking FFmpeg, even with hardware acceleration enabled, uses much more CPU (~2x-3x) than just using cv::cudacodec::VideoWriter.
VLC plays them just fine (if it has a .h264 extension it expects a h264 file, expecting it to play a raw h264 file with an mp4 extension as @mamsds did is asking a little too much), you just can’t seek effectively as there is no index.