Use of ROI for object tracking in video fails to write to file

Python
1

Problem

I am using OpenCV to process video from a security camera in order to draw a box around objects in motion and filter out parts of the video where there is no motion detected.

The issue is when I process the full frame, it works, but unwanted motion is detected. Such as, the swaying of branches in the wind. In order to resolve this, I created a ROI, and then used the ROI to process motion detection, but this is not working. When a ROI is used, the end result is a 258 byte file, which is obviously not a valid video file.

To avoid posting the entire section of code, I am just going to paste the relevant bit. Of course, this does not include extraction of the first frame to begin the comparison process.

ret, frame = cap.read()
        if not ret:
            log.info('Skipped frame')
            skipped_frames += 1
            if skipped_frames >= Options.skip_thres:
                log.info('Skipped threshold reached! Moving on to next video!')
                break
            else:
                log.info('No frames grabbed!')
                continue
        else:
            skipped_frames = 0
            frame_count += 1
        frame_time = cap.get(cv.CAP_PROP_POS_MSEC)
        log.debug('Frame is type: {}'.format(type(frame))) # Returns: ndarray
        frame = frame[int(ROI[1]):int(ROI[1]+ROI[3]),
                      int(ROI[0]):int(ROI[0]+ROI[2])]
        log.debug('Frame is now type: {}'.format(type(frame))) # Returns: ndarray
        next_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        frame_gray = cv.GaussianBlur(next_gray, (21, 21), 0)
        if first_frame is None:
            log.debug('First frame is None!')
            first_frame = old_gray
        else:
            counter_delay += 1
        if counter_delay > Options.delay_thres:
            counter_delay = 0
            first_frame = old_gray
        next_frame = frame_gray
        gray_diff = cv.absdiff(first_frame, next_frame)
        fg_mask = mog.apply(gray_diff, None, 0.01)
        kernel = cv.getStructuringElement(cv.MORPH_ELLIPSE, (3, 3))
        mask_thresh = cv.threshold(fg_mask, 180, 255, cv.THRESH_BINARY)[1]
        morphmask = cv.morphologyEx(mask_thresh, cv.MORPH_OPEN, kernel)
        contours, hierarchy = cv.findContours(morphmask,
                                              cv.RETR_EXTERNAL,
                                              cv.CHAIN_APPROX_SIMPLE)
       (...)
       vid_writer.write(contour_frame)

The problem might be with the approach taken to write the result to file. Instead of attempting to directly write the ROI to file, as if it was the same as a zoomed video. It might be needed to overlay the ROI back on top of the original frame, and then write the resulting merged frame to file.

2

My preliminary assumption was the problem arose from converting the MatLike type data into a Numpy.ndarray type array while performing the slice to generate the ROI. This assumption was derived from the documentation indicating opencv.cvtColor(":MatLike Object:", cv.COLOR_BGR2GRAY) -> MatLike Object, that is, the function cvtColor accepts a MatLike Object and returns a MatLike object.

3

it’s actually pretty simple, - if you open a VideoWriter with a certain Size,
you must write frames with EXACTLY that size to it later, else your images get discarded.

so, if you want to write a ROI, use (ROI[2], ROI[3]) as size param, NOT the actual input video size

4

So, you are saying it got buggered up when the video writer instance was created with different frame dimensions, than those that would be eventually used to write the video to.

That would make a big difference. Well, jiminy crickets!