Cropping white from image goes wrong :(

[Johan_Antonissen]

Hello there,

I’m trying to crop white from images generated in vpython. I’ve got the code working but am experiencing some weird behavior I do not understand.

An example is this image, i would like it to be cropped such that all the unnececairy white space is removed.

Unfortunatley the image crop is produced wrong. I wanted to post the result here but since im new I can only post one image. My problem is that a lot of the red arrows and red text are cut off, this should not happen.

I have a hard time understanding why this happens, i’ve tried to change the treshold below 254 but that did not work. I also tried to change the colors of the red arrow but that also did not work.

I would figure this should be extremely simple since my background is just 1 color.

Can anyone please explain me what is going on ?

This is my code:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Sat Jul 30 21:26:56 2022

@author: windhoos
"""

import cv2
import numpy as np
#import sys
import os

# get the path/directory
folder_dir = os.path.dirname(__file__)

for images in os.listdir(folder_dir):
 
    # check if the image ends with png
    if (images.endswith(".png")):
        
        img = cv2.imread(images)
        
        info = np.iinfo(img.dtype) # Get the information of the incoming image type
        img = img.astype(np.float64) / info.max # normalize the data to 0 - 1
        img = 255 * img # Now scale by 255
        img = img.astype(np.uint8)

        ## (1) Convert to gray, and threshold
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        th, threshed = cv2.threshold(gray, 254, 255, cv2.THRESH_BINARY_INV)
        
        ## (2) Morph-op to remove noise
        kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
        morphed = cv2.morphologyEx(threshed, cv2.MORPH_CLOSE, kernel)
        
        ## (3) Find the max-area contour
        cnts = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
        cnt = sorted(cnts, key=cv2.contourArea)[-1]
        
        ## (4) Crop and save it
        x,y,w,h = cv2.boundingRect(cnt)
        dst = img[y:y+h, x:x+w]
        cv2.imwrite('cv_'+images, dst)

This code is adopted from here.

Thank you very much in advance. Sorry for the probably noob question

[crackwitz]

adopted? okay… what parts of it do you understand, and what parts don’t you? how did you examine the code and its behavior? how did you look at intermediate values?

why threshold at level 254? for jpegs that may catch some noise (it doesn’t here)

why find contours, why only pick the largest one? you have multiple connected components. why not pass the entire mask to boundingRect?

[Johan_Antonissen]

Hello there,

Thank you for your reply.

My approach has been the follwing:
Firstly i did some basic research in openCV, i learnt that images are converted to matrices with numbers ranging from 0 to 255. My assumption was that since I have a computer generated image i should have a background thats all the same number without any variotion.

When I inspected the code i found that there is a treshold that i can input to detect the color change. With my current understanding, and by inspecting the matrices I found that 255 is the color white. So if i put the cropping treshold at 254 i should be able to find the boundaries when color begins. Ive also tried 1,255 49,255 and 150,255 but no effect.

I’ve also learnt from the code that the image is being converted from RBG to RGB and then to BW. I know that when a color spectrum is made black and white there is no difference between red and white. Therefore i changed the arrow color to blue. Unfortunatley that did not work.

Currently im reserachting what morphology is because i think the problem lies here. But this has been unsuccesful

As for your final question, I actually have no idea what that means, I would assume the whole rectangle that overlays the image instead of a bounding contour? It would be really helpful if i could get some more information on this.

Thank you again.

[crackwitz]

@Johan_Antonissen

since I have a computer generated image i should have a background thats all the same number without any variotion.

yes, but JPEG is lossy compression, and you can get some deviations in blocks (jpeg encodes pics in blocks) that aren’t entirely one color (because those are trivial to encode perfectly).

@Johan_Antonissen

Ive also tried 1,255 49,255 and 150,255 but no effect.

imagine a histogram. you’d want your threshold to sit nicely between “modes” (hills). cutting it too close can give you trouble, generally.

RGB:

grayscale:

@Johan_Antonissen

I know that when a color spectrum is made black and white there is no difference between red and white.

no. there is. depending on the conversion, (pure) red ends up as dark gray, not white.

anyway, you desire to segment your picture into white vs. non-white, so if red ends up as gray, that’s fine.

@Johan_Antonissen

I would assume the whole rectangle that overlays the image instead of a bounding contour?

no need to assume. try it. boundingRect, applied to a mask image, will give you the bounding rectangle of the nonzero pixels.

[Johan_Antonissen]

Ah I think I found the problem. Ny problem is that I have multiple bounding rectangles and I am selecting the biggest. What I need to do ia combining those rectangles to a single rectangle.

I’ve found a solution here:

How to merge nearby rectangles - OpenCV Q&A Forum

So I need to add something like this:

arr = []
for x,y,w,h in contourRects:
      arr.append((x,y))
      arr.append((x+w,y+h))

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners

[crackwitz]

nooo, that’s a hack. DO NOT find contours. it’s a needless extra step. I said all of that before.

[Johan_Antonissen]

Hello there,

Thank you for your reply.

Are you being sarcastic?
Your reply sounds like my computer will explode James Bond style when I do this

Combining the areas would be a great solution wouldn’t it?

[crackwitz]

it’s a solution. it’s okay but… you could just stick the mask in the boundingRect call and get all that in a single step that’s more efficient too.

also… minAreaRect gives you a “rotated rect”, so it’s probably no longer axis-aligned, so it’s a worse situation for cropping than just calling boundingRect

[Johan_Antonissen]

F*CK YEAH found my solution.

Thank you for helping! It helped a lot!

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Sat Jul 30 21:26:56 2022

@author: windhoos
"""

import cv2
import numpy as np
#import sys
import os

# get the path/directory
folder_dir = os.path.dirname(__file__)

for images in os.listdir(folder_dir):
 
    # check if the image ends with png
    if (images.endswith(".png")):
        
        img = cv2.imread(images)
        
        info = np.iinfo(img.dtype) # Get the information of the incoming image type
        img = img.astype(np.float64) / info.max # normalize the data to 0 - 1
        img = 255 * img # Now scale by 255
        img = img.astype(np.uint8)
        ## (1) Convert to gray, and threshold
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        th, threshed = cv2.threshold(gray, 120, 255, cv2.THRESH_BINARY_INV)
        
        ## (2) Morph-op to remove noise
        kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
        morphed = cv2.morphologyEx(threshed, cv2.MORPH_CLOSE, kernel)
        
        ## (3) Find contours
        contours, hierarchy = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        
        xmin=[]
        xmax=[]
        ymin=[]
        ymax=[]
        for c in contours:
            x,y,w,h = cv2.boundingRect(c)
            xmin.append(x)
            xmax.append(x+w)
            ymin.append(y)
            ymax.append(y+h)
            
        xmin=min(xmin)
        xmax=max(xmax)
        ymin=min(ymin)
        ymax=max(ymax)
        cv2.rectangle(img, (xmin, ymin), (xmax, ymax), (0,0,0), 2)
        dst = img[ymin:ymax, xmin:xmax]
        cv2.imwrite('cv_'+images, dst)