(JS) grabCut to findContours understanding masking

,
1

I’m trying to get a polygon out of grabcut.

The code below works as the example shows for grabcut

image
image946×1288 66.9 KB

However when I pass it through findContours it always returns the region of interest. eg it’s the exact bounding box in rect and ignores the mask.

Similar code to this works fine using a different process to create the mask. In that other process the value are only [0, 1].
I noticed that for the renderer it checks [0, 2].

Is it potentially something funny with how the masking works?
Do I need to cast it in someway?

let mask = new cv.Mat();
  let bgdModel = new cv.Mat();
  let fgdModel = new cv.Mat();
  let instance = {...data[0]}
  //console.log(instance)
  let rect = new cv.Rect(instance.x_min, instance.y_min, 
                         instance.width, instance.height);	// must be smaller then canvas
  cv.grabCut(src, mask, rect, bgdModel, fgdModel, 5, cv.GC_INIT_WITH_RECT);
  var hierarchy = new cv.Mat()
  var contours = new cv.MatVector()
  console.log(mask)
  cv.findContours(mask, contours, hierarchy, 
                  cv.RETR_CCOMP, 
                  cv.CHAIN_APPROX_SIMPLE);
  let contour = contours.get(0)  
  let epsilon = 0.01 * cv.arcLength(contour,true)
  let tmp = new cv.Mat();
  cv.approxPolyDP(contour, tmp, epsilon, true);
  polypoints = tmp.data32S
  console.log(polypoints, tmp.size())
 
  for (let i = 0; i < src.rows; i++) {
      for (let j = 0; j < src.cols; j++) {
          if (mask.ucharPtr(i, j)[0] == 0 || mask.ucharPtr(i, j)[0] == 2) {
              src.ucharPtr(i, j)[0] = 0;
              src.ucharPtr(i, j)[1] = 0;
              src.ucharPtr(i, j)[2] = 0;
          }
      }
  }
2

@anthony

I’m don’t have that experience with grabcut, I used it a little some years ago.

There is a tutorial in Python
https://docs.opencv.org/master/d8/d83/tutorial_py_grabcut.html

findContours requires a one channel binary image, say pixel with 0 are background, non 0 are foreground.

https://docs.opencv.org/master/d3/dc0/group__imgproc__shape.html#gadf1ad6a0b82947fa1fe3c3d497f260e0

May be the problem is you are looking for contour on a rgb image.

You already know the following sentence:
GrabCut uses 4 values as input mask: 0 and 2 are for background. I don’t know if it uses the same values for output mask, but look for mask2 in the tutorial code, you can see it substituting all 2 with 0 in the output mask.

Perhaps it would be best to

  • make a binary mask with only values 0 and 255, so you can imshow it
  • findContour on that mask
  • apply the mask to the image with AND
3

@Alejandro_Silvestri
Thank you very much that was the ticket

trick was to set values 2 → 0 eg:

  for (let i = 0; i < src.rows; i++) {
      for (let j = 0; j < src.cols; j++) {
          if (mask.ucharPtr(i, j)[0] == 2) {
            mask.ucharPtr(i, j)[0] = 0
          }
      }
  }

image

1 Like