LBF model in opencv.js not working

, ,
1

Hello fellow reader,

I am getting desperate about this topic. Unfortunately I am a beginner with Javascript, but my goal is to convert a Python script I have into a working web application, which should detect a face and draw points for 68 facial landmarks.

The crux is that opencv 4.8.0 does not work properly.
Its hard for me to find information about it (help is welcome) but i try to run the following in my javascript:

// Load the face landmark detector model
const landmarkDetector = new cv.FacemarkLBF();
landmarkDetector.loadModel("lbfmodel.yaml");

I get the following error:
‘TypeError: cv.FacemarkLBF is not a constructor’.

Unfortunately it does not work for me to build opencv.js (maybe have not spend enough time with it). But a friend tried to follow this post: [OpenCV.js] Build tracking module in opencv_contrib but encounter generate error to fix the error, but then opencv.js did not work properly anymore (the build version is 4.5.1).

I have uploaded all the relevant things to mit github, so feel free to take a look there:

There you find my working python example and my current try in Landmark_v1.html, as well as all necessary files (i hope)

Any help would be highly appreciated and please notify me if any important information is missing.

2

maybe you’re out of luck here.
from js, you can only use functions from the whitelist here –

and there is no face module (which also would require contrib modules)

you can try to edit the whitelist, add some modules / classes, wrangle the cmake options, and build it from src
(well, good luck, imo that’s difficult)

[edit] can you please explain, what you want to do with it ?
maybe we can find some alternative dnn for this ,…

3

Thank you for your answer!
Unfortunately I was sensing something like this already. Do you maybe have some resource on how you would add the modules manually? or do you think there will be an official release containing it soon?

As for my actual goal: I want to build a webssite that uses the client webcam to track a human face and detect facial landmarks, so I can later implement smile and/or blink detection. It is inspired by this program:
https://tastenkunst.github.io/brfv5-browser/

and to archieve that I only wanted to uses openCV without additional dependencies.

4

it hasnt changed in the last 2 or so years, also it’s already ‘fat enough’, no ? i guess, devs are pretty reluctant to add more,
or even, for ‘face’ making the contrib modules mandatory

shame, example browser not working here…
and the most interesting code parts are probably hidden here:

import { trackCamera, trackImage }
5

I see. I feel like OpenCV.js is currently not really cared about.
Do you have more information how one could do it by themselves?

Yes this program is selled commercially, so they won’t publish their secret sauce.

1 Like