Template matching with video input

I have a pretty basic template match function that takes a global variable, “path”, as input.

def operation(self):
    searchIn = cv2.imread(path, cv2.IMREAD_UNCHANGED)
    searchFor = cv2.imread('needle.png', cv2.IMREAD_UNCHANGED)

    result = cv2.matchTemplate(searchIn, searchFor, cv2.TM_CCOEFF_NORMED)

    cv2.imshow('Result', result)
    cv2.waitKey()
    cv2.destroyAllWindows()

    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)

    print(max_loc)

    print(max_val)

    if max_val > 0.9:
        print("Found it")

However, I want this function to take video input instead of just an image and I’m struggling with the implementation. I have tried something but keep getting an error message (see end of post for details). I don’t really know what I’m doing so if anyone could link me to some resources or explain how to do this, it would be really appreciated!

def operation(self):
        cap = cv2.VideoCapture(path)
        searchFor = cv2.imread('sampleIMG1.png', cv2.IMREAD_UNCHANGED)

        while cap.isOpened():
            ret, frame = cap.read()

            if not ret:
                print("Can't receive frame (stream end?). Exiting ...")
                break

            result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
            
            min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
            
            print(max_loc)
            print(max_val)
            if max_val > 0.9:
                print("Found it")

            if cv.waitKey(1) == ord('q'):
                break
            
        cap.release()
        cv.destroyAllWindows()

Here is the error message which I get:

  File "main.py", line 75, in operation
    result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'matchTemplate'
> Overload resolution failed:
>  - image is not a numpy array, neither a scalar
>  - Expected Ptr<cv::UMat> for argument 'image'
1 Like

hey @Shocks, as a rule of thumb:
everyone hates noobs posting images of text anywhere.
please replace with, uhmmm, TEXT, thank you
(and no fear, we’ll help with the formatting)

ah okay thanks for the advice, I will edit my post!

2 Likes

do you think the errors you got are useful to know?

(post deleted by author)

Yes, would make sense to include them, sorry. Here they are but I will also include them in the main post now!

  File "main.py", line 75, in operation
    result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'matchTemplate'
> Overload resolution failed:
>  - image is not a numpy array, neither a scalar
>  - Expected Ptr<cv::UMat> for argument 'image'

and you pass that… when matchTemplate expects an image

is there a way to pass a video into matchTemplate or only images?

at the risk of stating the obvious, you already have a loop that reads frames from your video… use them

ah, right I’m not using the loop at all there. If I use “frame” instead of “cap” in the matchTemplate I get a different error.

  File "main.py", line 75, in operation
    result = cv2.matchTemplate(frame, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-c2l3r8zm\opencv\modules\imgproc\src\templmatch.cpp:1164: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'

yes, matchTemplate expects single channel (grayscale) images, not color. use cvtColor

Added it and the video is displayed in grayscale but still getting the same error message.

grayFrame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

#cv2.imshow('video gray', grayFrame)
            
result = cv2.matchTemplate(grayFrame, searchFor, cv2.TM_CCOEFF_NORMED)

Error message:

 File "main.py", line 79, in operation
    result = cv2.matchTemplate(grayFrame, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-c2l3r8zm\opencv\modules\imgproc\src\templmatch.cpp:1164: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'

that goes for both arguments of course

1 Like

Yes that works! Thank you so much for your help, I really appreciate it.

Just for my understanding how is it that I can use templateMatch on images that have color but once I pass frames from a video in, I have to grayscale both input and ‘the needle’?

oh, interesting, both being 3-channel actually works… I just tried it.

anyway, both haystack and needle have to have the same number of channels. you specified IMREAD_UNCHANGED and that may read a grayscale image as grayscale.

it has nothing to do with where the frames came from, only what shape and type they are.

look at the .shape and .dtype of both the arguments you supplied originally (before cvtColor)

1 Like

ah, I see, that would explain it. Thank you again :slight_smile: