(JS) grabCut to findContours understanding masking

I’m trying to get a polygon out of grabcut.

The code below works as the example shows for grabcut

However when I pass it through findContours it always returns the region of interest. eg it’s the exact bounding box in rect and ignores the mask.

Similar code to this works fine using a different process to create the mask. In that other process the value are only [0, 1].
I noticed that for the renderer it checks [0, 2].

Is it potentially something funny with how the masking works?
Do I need to cast it in someway?

let mask = new cv.Mat();
  let bgdModel = new cv.Mat();
  let fgdModel = new cv.Mat();
  let instance = {...data[0]}
  let rect = new cv.Rect(instance.x_min, instance.y_min, 
                         instance.width, instance.height);	// must be smaller then canvas
  cv.grabCut(src, mask, rect, bgdModel, fgdModel, 5, cv.GC_INIT_WITH_RECT);
  var hierarchy = new cv.Mat()
  var contours = new cv.MatVector()
  cv.findContours(mask, contours, hierarchy, 
  let contour = contours.get(0)  
  let epsilon = 0.01 * cv.arcLength(contour,true)
  let tmp = new cv.Mat();
  cv.approxPolyDP(contour, tmp, epsilon, true);
  polypoints = tmp.data32S
  console.log(polypoints, tmp.size())
  for (let i = 0; i < src.rows; i++) {
      for (let j = 0; j < src.cols; j++) {
          if (mask.ucharPtr(i, j)[0] == 0 || mask.ucharPtr(i, j)[0] == 2) {
              src.ucharPtr(i, j)[0] = 0;
              src.ucharPtr(i, j)[1] = 0;
              src.ucharPtr(i, j)[2] = 0;


I’m don’t have that experience with grabcut, I used it a little some years ago.

There is a tutorial in Python

findContours requires a one channel binary image, say pixel with 0 are background, non 0 are foreground.


May be the problem is you are looking for contour on a rgb image.

You already know the following sentence:
GrabCut uses 4 values as input mask: 0 and 2 are for background. I don’t know if it uses the same values for output mask, but look for mask2 in the tutorial code, you can see it substituting all 2 with 0 in the output mask.

Perhaps it would be best to

  • make a binary mask with only values 0 and 255, so you can imshow it
  • findContour on that mask
  • apply the mask to the image with AND

Thank you very much that was the ticket

trick was to set values 2 → 0 eg:

  for (let i = 0; i < src.rows; i++) {
      for (let j = 0; j < src.cols; j++) {
          if (mask.ucharPtr(i, j)[0] == 2) {
            mask.ucharPtr(i, j)[0] = 0


1 Like