Suggestions for motion detection method

I’m still just learning opencv so I’m overwhelmed with all the different methods available to detect motion (frame differencing, background subtraction,…) so I’m looking here for suggestions.

I’m trying to build a small golf swing capture app (in python) that uses 2 cameras.

  • cam1 is focused on the golf ball lying on the floor
  • cam2 is further away and is ready to record the golfer’s movement

With cam1 I want to monitor a specific ROI (say 6 to 12 inches behind the ball) in the live video stream, and track the moving object (a golf club face) directly behind the golf ball. If the club head continues to move (say 6 inches) according to prediction, then I would trigger cam2 to start recording for a specified amount of time (say 2 seconds).

I already have the code to post-process a recorded swing using a Sports2D package (which in turn uses RTMLib) but I’m not sure what’s the best method to trigger my recording on.

starting a capture takes a moment.

in your place I’d record continuously and later discard what you don’t need.

and I hope your cameras have high enough frame rates. random info from google says golf clubs might move at 100 mph. if you had a 120 fps camera, the head would move 15 inches between frames.

I did that in my previous attempt at this project. Continually capture frames in the loop keeping track of frameno/timestamp, then I used pyaudio to detect the sound of club/ball-impact. Then saved the frames starting at impact-time - 1 second and ending at impact-time + .5 seconds, to a file.

This time I wanted to be smarter and avoid using audio and just trigger on detected movement.

I have something working but something is still off.
I am able to detect motion on cam1 by comparing frame by frame by calculating the % of pixels that changed within my ROI.
Then I trigger a capture on cam2 which saves to a file.

Here’s my problem
When I create a named window with a slider to view the generated capture,
somehow I’m getting 2 windows with a working slider on the wrong window?

Here’s the main section of my code

import cv2
import os
import sys
import time
import datetime
import logging
from contextlib import redirect_stderr
from light import Light
from camera import Camera
from motion_detection import MotionDetector
import threading

title_window = None
frames = []
framenum = 0
framecnt = 0

def load_frames(video_file):
    """Loads a video file as an array of frames."""
    cap = cv2.VideoCapture(video_file)
    global frames, framecnt
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        frames.append(frame)
    cap.release()
    framecnt = len(frames)

def on_change(value):
    global framenum, frames, framecnt
    if value < framecnt:
        cv2.imshow(title_window, frames[value])

def main():
    global framenum, frames, framecnt

    logging.basicConfig(filename='swing.log', format='%(asctime)s [%(threadName)s:%(thread)d] %(message)s', filemode='a', level=logging.INFO)
    logger = logging.getLogger(__name__)

    # Initialize lights
    light = Light(port="/dev/cu.usbmodem2401", baudrate=115200)
    light.status("ready")

    # Initialize cameras
    motion_trigger_camera = Camera(camera_index=1,fps=60.0, color_mode=cv2.COLOR_BGR2RGB, width=800, height=600, logger=logger)
    motion_response_camera = Camera(camera_index=0, fps=120.0, color_mode=cv2.COLOR_BGR2GRAY, width=800, height=600, logger=logger)

    # Start video streams
    motion_trigger_camera.start()
    motion_response_camera.start()

    # Initialize motion detector
    motion_detector = MotionDetector(roi=(420, 420, 300, 40), min_threshold=2.0, max_threshold=5.0, logger=logger)

    def record_swing(duration,outfile):
        try:
            logging.info(f"Recording swing to {swing_file}")
            motion_response_camera.start_recording(duration=duration, outfile=outfile)
        except Exception as e:
            logging.exception(f"Exception in recording thread: {e}")

    def view_swing(title_window,infile):
        global frames, framenum, framecnt
        try:
            load_frames(infile)
            on_change(0)
            logging.info(f"Loaded {framecnt} frames from {infile}")
            cv2.namedWindow(title_window)
            cv2.createTrackbar("Frame:", title_window , 0, framecnt, on_change)
            while True:
                cv2.imshow(title_window,frames[framenum])
                key = cv2.waitKey(1)
                if key == 99: # c key
                    break
                elif key == 2:  # Left arrow key
                    if framenum > 1:
                        framenum -= 1
                        cv2.setTrackbarPos("Frame:",title_window,framenum)
                elif key == 3:  # right arrow key
                    if framenum < (framecnt-1):
                        framenum += 1
                        cv2.setTrackbarPos("Frame:",title_window,framenum)
            cv2.destroyWindow(title_window)
        except Exception as e:
            logging.exception(f"Exception in viewing thread: {e}")
        
    try:
        while True:
            # Capture frames from both cameras
            tigger_frame = motion_trigger_camera.get_frame()
            swing_frame = motion_response_camera.get_frame()

            # Draw ROI rectangle on the frame
            x, y, w, h = motion_detector.roi
            cv2.rectangle(tigger_frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

            # Process the first camera frame for motion detection
            if motion_detector.detect_motion(tigger_frame):
                light.status("recording")

                # Start recording on a different thread for 2 seconds
                swing_file = "swing-" + datetime.datetime.now().strftime("%Y%m%d_%H%M%S") + ".mp4"
                recorder = threading.Thread(target=record_swing(duration=2,outfile=swing_file), daemon=False, name='Recorder')
                recorder.start()
                time.sleep(1)
                # block the main thread a lil bit
                #motion_response_camera.start_recording(duration=2,outfile=swing_file)
                light.status("busy")

                # show the recording
                if os.path.exists(swing_file):
                    viewer = threading.Thread(target=view_swing(title_window=f"swing: {swing_file}",infile=swing_file), daemon=True, name='Viewer')
                    viewer.start()
                light.status("ready")

            # Display the frames (optional)
            cv2.imshow('Trigger Camera', tigger_frame)
            #cv2.imshow('Swing Camera', swing_frame)

            # Break the loop on 'q' key press
            key = cv2.waitKey(1)
            if key == 113:
                break
    finally:
        # Release cameras and close windows
        motion_trigger_camera.release()
        motion_response_camera.release()
        light.close()
        cv2.destroyAllWindows()

def redirect_to_null(error_code, error_message, func_name, file_name, line_number):
    pass

if __name__ == "__main__":
    with open("error.log", "w") as f:
        sys.stderr = f
        cv2.redirectError(redirect_to_null)
        with redirect_stderr(f):
            main()

And the motion_detection part to calculate the movement_percentage is:

    def detect_motion(self, frame):
        # Convert frame to grayscale
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        # Crop the region of interest
        x, y, w, h = self.roi
        roi_gray = gray[y:y+h, x:x+w]

        # Initialize previous_frame if it's None
        if self.previous_frame is None:
            self.previous_frame = roi_gray
            return False

        # Compute the absolute difference between the current frame and the previous frame
        frame_diff = cv2.absdiff(self.previous_frame, roi_gray)
        self.previous_frame = roi_gray

        # Threshold the difference
        _, thresh = cv2.threshold(frame_diff, 25, 255, cv2.THRESH_BINARY)

        # Calculate the percentage of changed pixels
        non_zero_count = cv2.countNonZero(thresh)
        total_pixels = thresh.size
        movement_percentage = (non_zero_count / total_pixels) * 100
        if (self.min_threshold < movement_percentage < self.max_threshold):
            if self.logger:
                self.logger.info(f"Movement percentage: {movement_percentage:.2f}")```

I corrected the calls to start the recording and viewing separate threads
but now I’m getting an "Unknown C++ exception from OpenCV code "

The capture thread actually works, the mentioned file is created and viewable.

recorder = threading.Thread(target=record_swing, args=(swing_file,), daemon=False, name='Recorder').start()

viewer = threading.Thread(target=view_swing, args=(swing_file,), daemon=True, name='Viewer').start()

From mog logging I see seapare thread id/names but this error now:

2024-12-19 13:34:10,185 [MainThread:8384133184] Movement percentage: 4.58
2024-12-19 13:34:10,185 [Recorder:12901707776] Recording swing to swing-20241219_133410.mp4
2024-12-19 13:34:10,185 [Recorder:12901707776] Capturing to swing-20241219_133410.mp4
2024-12-19 13:34:10,185 [Recorder:12901707776] Recording complete
2024-12-19 13:34:10,325 [Viewer:6172438528] Loaded 241 frames from swing-20241219_133340.mp4
2024-12-19 13:34:10,325 [Viewer:6172438528] Exception in viewing thread: Unknown C++ exception from OpenCV code
Traceback (most recent call last):
  File "/Users/golf/Workspaces/motion-detection/golf-club-tracking-app/src/main.py", line 82, in view_swing
    cv2.namedWindow(title_window)
cv2.error: Unknown C++ exception from OpenCV code

that tends to happen when GUI calls happen from multiple threads, or even just a thread that’s not the main thread.

Thanks, I guess OpenCV doesn’t do well with threads.

My viewer named window works fine when I run it from the main thread.
I also fixed the trackbar not showing up in the correct named window.

I just have 2 issues

  1. How do I suppress the cap_gstreamer warnings?
    [ WARN:0@0.060] global cap_gstreamer.cpp:1173 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
  2. The WINDOW_GUI_EXPANDED flag isn’t working for me I create window with:
        cv2.namedWindow(viewer_window_name, flags=cv2.WINDOW_GUI_EXPANDED)
        cv2.resizeWindow(viewer_window_name, config['swing_width'], config['swing_height'])
        cv2.createTrackbar(trackbar_name, viewer_window_name , 0, framecnt, on_change)
        cv2.setTrackbarPos(trackbar_name,viewer_window_name,0)
        cv2.setTrackbarMin(trackbar_name,viewer_window_name,0)
        cv2.setTrackbarMax(trackbar_name,viewer_window_name,framecnt-1)
        cv2.moveWindow(viewer_window_name, 0, 400)

Resolved the QT issue by rebuilding opencv 4.10 from source with QT6.
I installed qt5 previously but I guess I didn’t compile opencv WITH_QT=ON last time.

Only remaining issue I have is the GStreamer: pipeline have not been created warnings. Any ideas?