TemplateMatch - Detecting Valorant (game) kills from Player

Python
1

Hi,

I’m new to OpenCV and Python but I managed to put together a piece of code that is supposed to detect a template (PNG image with transparency) in video frames of a captured video stream.

I’m using this picture as the template - I captured it from a video frame.

As I know that the kill in the game shows up on the top right corner, I’m not processing the full dimensions of each frame to make processing faster:

For instance, in this frame is when the kill happens and the template should’ve matched:

Capture4
Capture4639×683 180 KB

My current code is as following:

import numpy as np
import cv2 as cv

template = cv.imread('kill_frame5.png')

gray_template = cv.cvtColor(template, cv.COLOR_BGR2GRAY)

w, h = template.shape[:-1]

ret, mask = cv.threshold(gray_template, 200, 255, cv.THRESH_BINARY)

mask_inv = cv.bitwise_not(mask)
mask_inv = cv.cvtColor(mask_inv, cv.COLOR_GRAY2RGB)

cv.imshow('template', template)
cv.imshow('mask_inv', mask_inv)

# All the 6 methods for comparison in a list
'''
 methods = ['cv.TM_CCOEFF', 'cv.TM_CCOEFF_NORMED', 'cv.TM_CCORR',
            'cv.TM_CCORR_NORMED', 'cv.TM_SQDIFF', 'cv.TM_SQDIFF_NORMED']
'''

methods = ['cv.TM_CCORR_NORMED']

cap = cv.VideoCapture('video.mp4')

threshold = 0.8

while cap.isOpened():
    ret, frame = cap.read()

    cropped_frame = frame[:200, frame.shape[1] - 500:]

    # If frame is read correctly ret is True
    if not ret:
        print("Can't receive frame (stream end?). Exiting ...")
        break

    for meth in methods:
        method = eval(meth)
        res = cv.matchTemplate(cropped_frame, template, cv.TM_CCOEFF_NORMED, None, mask=mask_inv)

        loc = np.where( res >= threshold)
        for pt in zip(*loc[::-1]):
            cv.rectangle(cropped_frame, pt, (pt[0] + w, pt[1] + h), (0,255,0), 2)

    cv.imshow('frame', cropped_frame)

    if cv.waitKey(1) == ord('q'):
        break

cap.release()
cv.destroyAllWindows()

As I can’t attach all the assets as I’m new to the forum, I attached them here.

I’m sorry if this is a silly question, any help would be appreciated.

Thanks in advance!
Caio

2

it’s hard to tell from your screenshot of a screenshot, but do you use a full-resolution template and a downscaled “haystack”? then that is the error. matchTemplate does not do any multiscale detection.

my latest writeup and some more:

use TM_SQDIFF, never TM_CCORR*.

all these useless youtube tutorials show the same bad code and these youtubers never understand what it does, so they can’t explain what it does, or why it sucks.

1 Like
3

Thanks! I will make some tests later today after reading your links and see what can be improved. Probably it is a scale problem as you described.

4

This link helped me solving the problem. Thank you!