I just tested this against OpenCV 4.0.0 (75ed282). The same behaviour, so apparently this isn’t a new situation. But also, it makes me question if I’m doing something wrong, rather than it being a bug. Thoughts anyone?
those are going to be CV_8U types and addition wraps around. see if this expression works: (frameA >> 1) + (frameB >> 1). that’s distributing the 0.5* into the parenthesis and then transforming it (floating point!) into a cheap integer operation.
regarding the issue:
what backend do you get? ffmpeg? gstreamer? nail it down using apiPreference argument to videocapture and videowriter constructors
how much memory is approximately lost per round? what resolution do your videos have? to be a true memory leak, and not just an allocator being lazy about releasing the memory back to the OS, consumption must grow arbitrarily with repeated object constructions+destructions.
Thanks for the advice on the expression! I will investigate and try to learn from your suggestion.
With respect to the issue, sorry I should have clarified the backend - especially since I was wondering if that was a consideration. By using getBackendName() I found that the readers are: MSMF, and the writer is: CV_MJPG. I confirmed this by explicitly setting the backend with these:
int apiPrefReaders = cv::CAP_MSMF;
int apiPrefWriter = cv::CAP_OPENCV_MJPEG;
Round 1: 15MB-95MB
Round 2: 95MB-187MB
Round 3: 187MB-257MB
So roughly ~80MB each time.
The video is 1280x720, and as you probably saw, MJPG @24fps.
I did try specifying DSHOW, FFMPEG, both read and write to be MSMF, VFW. These permutations so far either lead to no output file (nor CPU usage), or expected CPU usage and a 0 byte file result.
Thanks again for your input. Looking forward your thoughts on this.
For anyone who sees memory leak looking stuff with Windows 10 and OpenCV 4.0.0/ 4.51 / 4.52 when using videocapture, you may have a similar situation to me. That is to say, MSoft MediaFoundation as a backend for the reader might not be behaving as well as it should.
Instead, explicitly set your apiBackend for the reader:
Not only do I see no leaked memory, but it’s actually a little faster.
On the topic of the expression:
I did try this but it’s not a syntax that I can get away with in c++. I tried a few things but I was unable to get a version of your suggestion to work. Would you mind explaining a bit more verbosely what a better alternative is to what I was doing please? Is your point that I’m combining a float operation on int matrices?
the idea is: if you have two pixels that are both 255 (uint8), and you add them, your result has to be uint8 again, but 510 doesn’t fit in that range. it’ll become 254. that’s how integers wrap around. then you *0.5 this and you’re left with 127.0… for two white pixels blended together.
halving both before the addition keeps the addition from overflowing. with a bit shift, or integer division by 2, you’d get 127 + 127 = 254 as the result. the exact mathematical result would have been 255.0, so this also has downsides.
or you convert both operands into CV_16U (CV_16UC3) before the calculation, which are 16 bit integers. no overflow there. or you convert both into CV_32F (CV_32FC3).
the choice depends on your desired result type, but also on resource usage. 32 bit floats use more memory than 8 or 16 bit integers. integer operations are faster to calculate than floating point operations.
This is an excellent explanation! Thanks very much for the comprehensive answer and sharing your thinking. I had thought that it would max out the 8bit value on overflow and imply a clamp on the values rather than wrapping around. I was focusing so much on the memory stuff I wasn’t even paying attention.
The bit shift option would have been cool, but all good - plenty of other options available.
Good point on reporting MSMF as a possible bug. I’ll carve out a more concise example and post it in github as suggested.