# Template matching with video input

**URL:** <https://forum.opencv.org/t/template-matching-with-video-input/6563>\
**Category:** Python\
**Tags:** highgui, matchtemplate, videoio\
**Created:** [December 4, 2021, 7:20pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563 "2021-12-04T19:20:15Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 4, 2021, 7:20pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/1 "2021-12-04T19:20:15Z")

</div>

I have a pretty basic template match function that takes a global variable, “path”, as input.

```auto
def operation(self):
    searchIn = cv2.imread(path, cv2.IMREAD_UNCHANGED)
    searchFor = cv2.imread('needle.png', cv2.IMREAD_UNCHANGED)

    result = cv2.matchTemplate(searchIn, searchFor, cv2.TM_CCOEFF_NORMED)

    cv2.imshow('Result', result)
    cv2.waitKey()
    cv2.destroyAllWindows()

    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)

    print(max_loc)

    print(max_val)

    if max_val > 0.9:
        print("Found it")

```

However, I want this function to take video input instead of just an image and I’m struggling with the implementation. I have tried something but keep getting an error message (see end of post for details). I don’t really know what I’m doing so if anyone could link me to some resources or explain how to do this, it would be really appreciated!

```auto
def operation(self):
        cap = cv2.VideoCapture(path)
        searchFor = cv2.imread('sampleIMG1.png', cv2.IMREAD_UNCHANGED)

        while cap.isOpened():
            ret, frame = cap.read()

            if not ret:
                print("Can't receive frame (stream end?). Exiting ...")
                break

            result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
            
            min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
            
            print(max_loc)
            print(max_val)
            if max_val > 0.9:
                print("Found it")

            if cv.waitKey(1) == ord('q'):
                break
            
        cap.release()
        cv.destroyAllWindows()

```

Here is the error message which I get:

```auto
  File "main.py", line 75, in operation
    result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'matchTemplate'
> Overload resolution failed:
> - image is not a numpy array, neither a scalar
> - Expected Ptr<cv::UMat> for argument 'image'

```

---

<div class="post-metadata">

**Author:** ![berak](https://avatars.discourse-cdn.com/v4/letter/b/85f322/32.png) [@berak](https://forum.opencv.org/u/berak)\
**Post date:** [December 4, 2021, 7:39pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/3 "2021-12-04T19:39:59Z")

</div>

hey @Shocks, as a rule of thumb:  
everyone hates noobs posting _images of text_ anywhere.  
please replace with, uhmmm, TEXT, thank you  
(and no fear, we’ll help with the formatting)

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 4, 2021, 8:13pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/4 "2021-12-04T20:13:12Z")

</div>

ah okay thanks for the advice, I will edit my post!

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 4, 2021, 9:01pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/5 "2021-12-04T21:01:21Z")

</div>

do you think the errors you got are useful to know?

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 4, 2021, 9:24pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/6 "2021-12-04T21:24:23Z")

</div>

(post deleted by author)

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 4, 2021, 9:25pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/7 "2021-12-04T21:25:20Z")

</div>

Yes, would make sense to include them, sorry. Here they are but I will also include them in the main post now!

```auto
  File "main.py", line 75, in operation
    result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'matchTemplate'
> Overload resolution failed:
> - image is not a numpy array, neither a scalar
> - Expected Ptr<cv::UMat> for argument 'image'

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 4, 2021, 11:43pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/8 "2021-12-04T23:43:29Z")

</div>

> [@Shocks](#):
>
> ```auto
> cap = cv2.VideoCapture(path)
> 
> ```

> [@Shocks](#):
>
> ```auto
> result = cv2.matchTemplate(cap, searchFor, cv2.TM_CCOEFF_NORMED)
> 
> ```

and you pass that… when `matchTemplate` expects an image

> [@Shocks](#):
>
> ```auto
> > Overload resolution failed:
> > - image is not a numpy array,
> 
> ```

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 4, 2021, 11:47pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/9 "2021-12-04T23:47:35Z")

</div>

is there a way to pass a video into matchTemplate or only images?

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 5, 2021, 12:39am UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/10 "2021-12-05T00:39:52Z")

</div>

at the risk of stating the obvious, you already have a loop that reads frames from your video… use them

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 5, 2021, 2:02pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/11 "2021-12-05T14:02:57Z")

</div>

ah, right I’m not using the loop at all there. If I use “frame” instead of “cap” in the matchTemplate I get a different error.

```auto
  File "main.py", line 75, in operation
    result = cv2.matchTemplate(frame, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-c2l3r8zm\opencv\modules\imgproc\src\templmatch.cpp:1164: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 5, 2021, 2:09pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/12 "2021-12-05T14:09:58Z")

</div>

yes, matchTemplate expects _single channel_ (grayscale) images, not color. use cvtColor

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 5, 2021, 2:28pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/13 "2021-12-05T14:28:09Z")

</div>

Added it and the video is displayed in grayscale but still getting the same error message.

```auto
grayFrame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

#cv2.imshow('video gray', grayFrame)
            
result = cv2.matchTemplate(grayFrame, searchFor, cv2.TM_CCOEFF_NORMED)

```

Error message:

```auto
 File "main.py", line 79, in operation
    result = cv2.matchTemplate(grayFrame, searchFor, cv2.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-c2l3r8zm\opencv\modules\imgproc\src\templmatch.cpp:1164: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 5, 2021, 3:14pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/14 "2021-12-05T15:14:44Z")

</div>

that goes for both arguments of course

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 5, 2021, 3:29pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/15 "2021-12-05T15:29:45Z")

</div>

Yes that works! Thank you so much for your help, I really appreciate it.

Just for my understanding how is it that I can use templateMatch on images that have color but once I pass frames from a video in, I have to grayscale both input and ‘the needle’?

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [December 5, 2021, 4:03pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/16 "2021-12-05T16:03:16Z")

</div>

oh, interesting, both being 3-channel actually works… I just tried it.

anyway, both haystack and needle have to have the same number of channels. you specified `IMREAD_UNCHANGED` and that may read a grayscale image as grayscale.

it has nothing to do with where the frames came from, only what shape and type they are.

look at the `.shape` and `.dtype` of both the arguments you supplied originally (before cvtColor)

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [December 5, 2021, 4:13pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/17 "2021-12-05T16:13:18Z")

</div>

ah, I see, that would explain it. Thank you again 🙂

---

<div class="post-metadata">

**Author:** ![Shocks](https://avatars.discourse-cdn.com/v4/letter/s/ecd19e/32.png) [@Shocks](https://forum.opencv.org/u/Shocks)\
**Post date:** [January 28, 2022, 8:37pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/18 "2022-01-28T20:37:50Z")

</div>

Currently, the video is read in real-time, which for my purposes is not very ideal. Is it possible to read the video faster?

Relevant code:

```auto
def operation(self):
        thread1 = Thread(target=self.testFunc)
        thread1.start()
        cap = cv2.VideoCapture(path)
        searchFor = cv2.imread('sampleIMG1.png', cv2.IMREAD_UNCHANGED)

        

        while cap.isOpened():
            ret, frame = cap.read()

            if not ret:
                print("Can't receive frame (stream end?). Exiting ...")
                stop_order.set()
                thread1.join()
                break

            global grayFrame
            grayFrame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
            graySearchFor = cv2.cvtColor(searchFor, cv2.COLOR_BGR2GRAY)

            #cv2.imshow('video gray', grayFrame)
            
            result = cv2.matchTemplate(grayFrame, graySearchFor, cv2.TM_CCOEFF_NORMED)
            
            min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)

            
            
            print(max_loc)
            print(max_val)
            if max_val > 0.9:
                print("Found it")
                result_available.set()
            elif max_val < 0.9:
                result_available.clear()

            if cv2.waitKey(1) == ord('q'):
                break

```

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [January 28, 2022, 10:31pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/19 "2022-01-28T22:31:38Z")

</div>

“in realtime”?

you do a lot of processing. matchTemplate is _not_ cheap. check CPU usage. how many cores worth of CPU time is used by the process?

---

<div class="post-metadata">

**Author:** ![Steve\_in\_Denver](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Steve\_in\_Denver](https://forum.opencv.org/u/Steve_in_Denver)\
**Post date:** [January 28, 2022, 11:01pm UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/20 "2022-01-28T23:01:03Z")

</div>

It sounds like you want to read the frames from the file faster than the framerate of the encoded video file, but when you call cap.read() it blocks until enough time has elapsed since the last read? I haven’t used a cv videoCapture object for reading video from file, but maybe you can set the FPS on the video device (and override the file’s encoded FPS)? Or maybe you can set the position in the sequence explicitly, and then call read()? (I’m hoping the read call would return immediately in this case.)

I’m just making guesses, though.

---

<div class="post-metadata">

**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\
**Post date:** [January 29, 2022, 12:45am UTC](https://forum.opencv.org/t/template-matching-with-video-input/6563/21 "2022-01-29T00:45:34Z")

</div>

> [@Steve\_in\_Denver](#):
>
> maybe you can set the FPS on the video device

impossible. OpenCV doesn’t look at FPS when reading video files. there is no mechanism that would “pace” the reading because nobody wants that. I’m sure you’ll get an error code when you even try to set such a property on a VideoCapture of a file.

the only backend that _is even capable_ of causing such shenanigans is gstreamer and even then one would need to do something explicitly and specifically wrong. I saw _one_ instance of such an issue in the past 5+ years. nobody else ever reported that again. probably because the person back then wrote their own gstreamer pipeline and made a mistake there.

the more likely explanation is that OP generated a hypothesis that’s close enough to reality (video only _appears_ to “play back” close to realtime), then latched onto the hypothesis without considering alternatives.

especially when you _haven’t_ used VideoCapture on files before, _please_ be careful with generating more unsupported hypotheses, or believing and _feeding into_ the first unsupported hypothesis. doesn’t help anybody, only adds confusion to the both of you, and any passers-by looking for answers that’ll find this later.

[Next page](https://forum.opencv.org/t/template-matching-with-video-input/6563.md?page=2)