Splitting the "G" in a Guinness - OpenCV analysis to detect point of beer line in ROI

Python
1

I’m working on an application that will take in an image of a Guinness glass, and analyze the beer to see how successful users are in “splitting the G”.

“Splitting The G” is when someone takes a full pint of Guinness, and takes a couple big gulp, so that when they put down the glass, the line of the beer directly crosses the middle of the “G” on the Guinness glass.

I’m using a Roboflow project that will take an image, and return the coordinatoes of the “G” in the photo. That successfully gives me a narrow ROI of what I’m looking for. What I’m currently using OpenCV for is determining the “score” of the G split – which right now is a number of how close the beer line is to the middle of my ROI – roughly where the “G” is.

What I’m currently struggling with is determining the best way to calculate the score given my scenario. I have a function that is using OpenCV to look in my ROI to find the beer line, to calculate how close my user is to splitting the G. Ideally – my ROI will be half black beer, below the “G”, and half “white foam”. I’m extracting a small column in the middle of my ROI, and finding the point in my column where the gradient value jumps the most – i.e. where the contrast is the greatest.

What I wanted to ask is – is this the best approach for this logic to calculate the beer/foam line? There are certain cases that are still coming up with incorrect scores, such as where the white font of the “G” interferes with the calculations, or there is no beer/foam line present. I wanted to check with others to see if this function seems to be the best way to do this, or if I was missing something else. I also trying masking the “G” before processing but wasn’t successful in that. Thanks in advance for reading and appreciate any and all help/comments

Here is the function that takes in the ROI and tries to find the foam line.

def detect_foam_line(grayscale_image, x1, y1, y2):
    """
    Detect the foam line by analyzing a narrow column down the middle of the ROI.

    Args:
    - grayscale_image: Grayscale image of the bounding box containing the "G".
    - x1: X-coordinate of the left side of the ROI.
    - y1: Y-coordinate of the top of the ROI.
    - y2: Y-coordinate of the bottom of the ROI.

    Returns:
    - foam_line_y: Y-coordinate of the foam line within the cropped ROI.
    """
    height, width = grayscale_image.shape
    center_x = width // 2

    # Define the 5-pixel-wide column around the center
    column_start = max(center_x - 2, 0)
    column_end = min(center_x + 3, width)

    # Extract the 5-pixel-wide column in the middle of the ROI
    narrow_column = grayscale_image[:, column_start:column_end]

    # Save the narrow column for debugging
    cv2.imwrite("narrow_column_debug.jpg", narrow_column)
    
    # Step 1: Calculate the vertical intensity profile
    vertical_profile = np.mean(narrow_column, axis=1)

    # Step 2: Calculate intensity gradient
    gradient = np.abs(np.diff(vertical_profile))

    # Save the gradient as an image for debugging
    gradient_image = np.expand_dims(gradient, axis=1)
    cv2.imwrite("gradient_debug.jpg", gradient_image)

    # Step 3: Check for a significant jump
    max_gradient = np.max(gradient)
    gradient_threshold = 40  # Adjust this threshold based on your data
    print(f"Max Gradient: {max_gradient:.2f}")

    if max_gradient < gradient_threshold:
        print("No significant intensity jump detected. Likely all foam or all beer.")
        raise ValueError("No significant intensity variation detected. Likely all foam or all beer.")
    
    # Apply Gaussian Blur to smooth out noise
    blurred_column = cv2.GaussianBlur(narrow_column, (3, 3), 0)

    # Apply Otsu's thresholding to dynamically binarize the image
    _, binary_column = cv2.threshold(blurred_column, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

    # Save the binary column for debugging
    cv2.imwrite("binary_column_debug.jpg", binary_column)
    

    # Calculate the vertical intensity profile (average across the 5-pixel width)
    binary_profile = np.mean(binary_column, axis=1)
    print("Binary Profile:", binary_profile)

    # Detect transition: Find where the change in intensity is the biggest
    gradient = np.abs(np.diff(binary_profile))
    foam_line_y = int(np.argmax(gradient)) + y1  # Offset by y1 to convert to global coordinates

    return foam_line_y

Here is an example ROI I’ll process:
roi_debug