I want opencv to recognize the numbers on the game screen

Python
, ,
1

I have an image of a certain game screen and would like to identify what number is shown on it.
These numbers are numbers 1-18.
So I prepared numbers 1-18 and did template mathing to measure the highest scoring number.
However, the accuracy of the matching result was about 50%.
Then I used AWS OCR and the accuracy is 80%.

How can I get opencv or any other method to determine the numbers?
Please answer.

Below is a sample program to measure with template mathing and the numbers we want to measure.
Originally, I wanted to include multiple samples of numbers and images of the target screen, but this site allows only one image to be uploaded, so I have only uploaded a sample of 18 numbers.

        matchs = []
        screen_num_img = cv2.imread('images/screenshot/sample.png')
        for i in range(1, 18):
            num_img = cv2.imread(f'images/round_images/{i}.png')
            num_img_gray = cv2.cvtColor(num_img, cv2.COLOR_BGR2GRAY)
            match = cv2.matchTemplate(image=screen_num_img, templ=num_img_gray, method=cv2.TM_CCOEFF_NORMED)
            _, maxVal, _, maxLoc = cv2.minMaxLoc(match)
            matchs.append({"value": maxVal, "loc": maxLoc, "num": i})
        max_match = max(matchs, key=lambda x: x["value"])
        return max_match["num"]

スクリーンショット 2024-04-02 150129

2

bad TM_* mode.

use one of the SQDIFF types. result is a distance, i.e. low is good, 0 is perfect.

are those numbers always displayed on the same color of background? if yes, you can absolutely expect a perfect match.

3

Thanks for the reply.
I tried with SQDIFF but did not get good results.
I wrote a program to line up the comparison images to answer your question, and found that the images are degraded considerably when reducing the image or getting the data from the source.
I will post a side-by-side of the target image and the template.
If they are so different, template matching will not give good results, right?

スクリーンショット 2024-04-02 234011

4

it does not do multiscale matching. looks like you expected it to do that.

I’m guessing here because you have not shown me actual data. your illustration appears to resize the pictures to be “same”.

given same-scale images, template matching could reasonably detect that, maybe.

it’s not the ultimate solution to every conceivable problem.

it’s a good solution to some problems.

some other problems can be made to fit. but you need to know this.

5

Thanks for the reply.
Sorry, this website limits the number of data you can upload to one.
If there are many similar images and they are degraded, it may be difficult to use template matching.
I will try to find a means to avoid degradation. Thank you very much.