I’m using a BackgroundSubtractorKNN to recognize moving objects in a video. Due to different light conditions there are many artifacts that I want to remove. Therefore I’m using a HOGDescriptor. Unfortunately the recognition is still poor. Many static objects are recognized as persons. Is there any to MS Teams background remove method in OpenCV?
you can test this code if you use c++ or share your code to test if we can help.
You can see the result in this example. The person is not recognized but other objects are marked.
BackgroundSubtractor backSub = Video.createBackgroundSubtractorKNN(4000,200,true);
VideoCapture capture = new VideoCapture(videoFileName);
Mat frame = new Mat();
Mat fgMask = new Mat();
HOGDescriptor hogDescriptor = new HOGDescriptor();
hogDescriptor.setSVMDetector(HOGDescriptor.getDefaultPeopleDetector());
capture.read(frame);
if (frame.empty()) {
break;
}
MatOfRect found = new MatOfRect();
MatOfDouble weight = new MatOfDouble();
Mat m3 = new Mat(frame.width(), frame.height(), CvType.CV_8UC1);
Imgproc.cvtColor(frame, m3, Imgproc.COLOR_BGRA2GRAY);
hogDescriptor.detectMultiScale(m3, found, weight, 0, new Size(8, 8),
new Size(32, 32), 1.05, 2, false);
Rect[] rects = found.toArray();
weight.release();
found.release();
if (rects.length > 0) {
for (int i = 0; i < rects.length; i++) {
if (rects[i].x < 0 || rects[i].x + rects[i].width > frame.width() || rects[i].y < 0 || rects[i].y + rects[i].height > frame.height())
continue;
Imgproc.rectangle(m3, new Point(rects[i].x, rects[i].y), new Point(rects[i].x + rects[i].width, rects[i].y + rects[i].height), new Scalar(255, 0, 0, 255), 2);
}
}
`it’s likely requiring the COMPLETE human to be in frame.
cascades will NOT help you achieve some kind of background removal/blur effect.
It’s not a problem of the input image but of the substractor of opencv . I used the webpage https://www.remove.bg/ that is producing nearly perfect results
well yes because that likely uses a DNN, for segmentation (not detection).
I added DNN for detection. Results are ok (lower and upper parts are clipped and some persons are doubled). But I only get the outer box of the objects. How substract the background?
if you’re already into dnn’s, look out for RCNN models, which can do segmentation as well
Thanks a lot. Looks promising
btw, looking back, i don’t see any of this in your code
My code is currently a real mess due it is a playground for finding the proper solution. So I only copied snippets
The part of background substractor is:
Mat blured = new Mat();
Imgproc.GaussianBlur(frame, blured, new Size(25, 25), 0);
Imgproc.threshold(blured,blured,15,255,0);
Mat hist = new Mat();
hist=blured;
List<Mat> channels = new ArrayList<Mat>();
Core.split(hist, channels);
Imgproc.equalizeHist(channels.get(0), channels.get(0));
Imgproc.equalizeHist(channels.get(1), channels.get(1));
Imgproc.equalizeHist(channels.get(2), channels.get(2));
Core.merge(channels, hist);
Imgproc.cvtColor(hist, hist, Imgproc.COLOR_YCrCb2BGR);
Imgproc.cvtColor(hist, blured, Imgproc.COLOR_RGB2HSV);
//Here the background substractor is applied
backSub.apply(blured, fgMask,0.3);i guess, you wanted something like this , which is running some UNet under the hood.
and btw, no you probably should not apply equalizeHist() on B,G,R channels, this will only lead to weird artefacts , if you merge it again (visualize output !).
better idea:
- convert to HSV (or similar) and split()
- equalizeHist() only the luminance channel (so the original color distribution stays intact)
- merge back and back to BGR