Suggestions for scoring a simple user drawing based on template?

Python
1

Hey Everyone,

OpenCV is amazing, but it’s a bit overwhelming with it’s extensive features to find the right one for your use case sometimes!. I was wondering if my approach to the following problem is the correct one, or if someone could point me into the right direction to solve it using OpenCV

The problem:
I have a simple shape, reduced to a few core points:

templateheart
templateheart720×720 11 KB

I then ask a user to attempt to draw the same shape in the same sized window,
As this user is doing this i note down positions at certain time interval. the end result looking something like this:
(sorry only allowed one embed as new user, so the other images will be links)

720×720 12.2 KB

I’d then like to give some arbitrary score of how “close” the user drew the shape to the template.

My current proposed solution doesn’t use opencv, but instead uses a probably way too convoluted solution of my own:

I break up the template image into more points, and I do the same for the user drawing

1440×720 37.7 KB

Now I check for each point in the user drawing how far they are from a point in the template, and give a penalty based on distance. (this penalizes if you draw parts that shouldn’t be there according to template)

I then do the opposite by checking for each point in the template how far they are from a point in the user’s drawing (this penalizes if you don’t draw parts that should be there according to template)

Then I add these scores together for some arbitrary “negative score”

Is this the right approach? or am I re-inventing the wheel?

Any feedback is much appreciated - Wilco

2

it looks quite like the hausdorf distance , which is also implemented in the shape module . have a look !

1 Like
3

yes, sounds like a sensible approach. Hausdorff considers the worst of all shortest distances. a good measure. I don’t know how much more you’d wanna calculate from complete nearest neighbor information for both sets. that could give the user detailed feedback on where they drew pretty accurately, or where they deviated entirely.

if the user doesn’t draw over the template, the result could be shifted, rotated, or even scaled. perhaps run a few iterations of Iterative Closest Point, to try to remove any shift or rotation, which would make the scoring less harsh… if there’s a chance the user drew something scaled up/down, compensating for that too maybe.

1 Like
4

Thanks for the pointers!

Hausdorff does indeed look very similar to what I’m doing! I’ll definitely look into the shape module and hopefully I’m able to decipher the c++ example

Good point on accounting for rotation/scale. I’m planning to give the user a set starting point so that will hopefully make it a little bit easier to account for shifting! but nonetheless applying IPC will definitely be worth it for rotation. I found this really detailed documentation that will help a bunch:

Any suggestions for existing methods to compensate for scale?

I wish I had more technical vocab! Hausdorff… ICP… exactly what I need and here I am googling different variations of the word “shape/drawing/matching/aligning” getting barely related topics to my problems

1 Like