OpenCV morphological dilation implementation might be wrong?

1

Hello,
I have been using OpenCV (in Python) for quite a while now, and have been using morphological filter for a long time. Today, I ran some code performing morphological dilations and erosions with a non symmetrical structuring element (not equal to itself when turned 180° around the origin), and found out that OpenCV results were not what was expected…

First, here is some code for morphological erosion:

import numpy
import cv2
el = [[0,1,1]]
image = [[2,3,5,4,6],[4,3,2,4,3],[3,4,3,2,0]]
r = cv2.erode(numpy.array(image, dtype=numpy.uint8), numpy.array(el, dtype=numpy.uint8)*255)
print(numpy.array(image, dtype=numpy.uint8))
print("***")
print(r)

The result is the following

[[2 3 5 4 6]
 [4 3 2 4 3]
 [3 4 3 2 0]]
***
[[2 3 4 4 6]
 [3 2 2 3 3]
 [3 3 2 0 0]]

The structuring element is actually simple: for each pixel, we consider the pixel itself and its right neighbor. Performing an erosion of the image with such structuring element gives the expected result: each pixel is replace by the minimum value of the pixel itself and its right neighbor. So, for the erosion, everything works as expected.

When performing the dilation, there is a problem…

r2 = cv2.dilate(numpy.array(image, dtype=numpy.uint8), numpy.array(el, dtype=numpy.uint8)*255)
print(numpy.array(image, dtype=numpy.uint8))
print("***")
print(r2)

The result is

[[2 3 5 4 6]
 [4 3 2 4 3]
 [3 4 3 2 0]]
***
[[3 5 5 6 6]
 [4 3 4 4 3]
 [4 4 3 2 0]]

The result looks like each pixel was replaced by the maximum value between the pixel and its right neighbor. However, it should not be that.

Dilation is defined as a maximum, but using the symmetric of the input structuring element. In fact, the structuring element should be turned 180° around the center by the dilation function before being used for computing a maximum… In this example, each pixel should be replaced by the maximum value between the pixel and its left neighbor. But it is not the case, as OpenCV dilate function seems to fail to perform this rotation.

The definition of morphological dilation can be found on various source (“Hands-on morphological Image Processing” by Dougherty and Lotufo for example, or Dilation (morphology) - Wikipedia for a more direct source), and the implementation of morphological dilation of OpenCV seems to fail to take into account this rotation of the structuring element.

To support this, the same code on Matlab with the image processing toolbox gives the expected result:

image = [2 3 5 4 6; 4 3 2 4 3; 3 4 3 2 0];
se = strel([0 1 1]);
r = imerode(image,se);
r2 = imdilate(image,se);
image
disp('***')
r
disp('***')
r2

The output is

image =

     2     3     5     4     6
     4     3     2     4     3
     3     4     3     2     0

***

r =

     2     3     4     4     6
     3     2     2     3     3
     3     3     2     0     0

***

r2 =

     2     3     5     5     6
     4     4     3     4     4
     3     4     4     3     2

Matlab and OpenCV have the same result for the erosion, but differ for the dilation…

The consequences are, for example, that some results of the book “Hands-on morphological Image Processing” cannot be reproduced (for example, Figure 1.9 of chapter 1).
Moreover, results of transformations relying on dilation are also false. For example, morphological opening should be anti-extensive (Opening (morphology) - Wikipedia), meaning that the result of the morphological opening of an image A by a structuring element B should be pixel-wise inferior or equal to A. And OpenCV fails to exhibit such property:

r3 = cv2.morphologyEx(numpy.array(image, dtype=numpy.uint8), cv2.MORPH_OPEN, numpy.array(el, dtype=numpy.uint8) * 255)
print(numpy.array(image, dtype=numpy.uint8))
print("***")
print(r3)

The output is

[[2 3 5 4 6]
 [4 3 2 4 3]
 [3 4 3 2 0]]
***
[[3 4 4 6 6]
 [3 2 3 3 3]
 [3 3 2 0 0]]

The top left pixel of r3 has a higher value that the top left pixel of the image. On Matlab, we don’t have such anomaly:

image = [2 3 5 4 6; 4 3 2 4 3; 3 4 3 2 0];
se = strel([0 1 1]);
r3 = imopen(image,se);
image
disp('***')
r3

The output is

image =

     2     3     5     4     6
     4     3     2     4     3
     3     4     3     2     0

***

r3 =

     2     3     4     4     6
     3     3     2     3     3
     3     3     3     2     0

Here, we see that the result of the opening transform respect the anti-extensive property: each pixel of the result if inferior or equal to the corresponding pixel of the input.

I am therefore quite convinced that OpenCV does not implement correctly the definition of the dilation. When using a symmetrical structuring element (disk, line, cross, square, …), the problem does not appear as rotating the structuring element 180° around its origin does not change the structuring element. However, when using non symmetrical structuring element, the problem arises…

I searched to see if this problem has already been brought up, but I couldn’t find anything on this matter… That’s why I am writing this topic today. If this is indeed an error of implementation of the dilation in OpenCV, how could the message be passed to the developers ?

Thanks for reading up to here!

edit : I am using opencv-python 4.7.0.72

2

it shouldn’t be flipped for one operation and not the other. either it’s flipped, which is then a convolution, or it’s not, so it’s a correlation.

what you mean is that you want its center to be shifted. that’s a matter of taste and not necessarily required inherently by either operation.

some people seem to be doing this shifting business to fix the offset caused by even-sized kernels, where the center doesn’t fall on an element exactly. so for one of erosion/dilation, they go one half left, and for the other operation they go one half right, and that cancels out.

if you want to shift the center, just pass an anchor.

I’d also recommend that you convert your arrays to numpy arrays before calling. makes the code neater.

import numpy as np
import cv2 as cv

el = np.uint8([[0,1,1]])
# this is lopsided, so there's no way around having to think about where the anchor is

im = np.zeros((3,5), dtype=np.uint8)
im[0, 1:2] = 1 # one set
im[1, 1:3] = 1 # two set
im[2, 1:4] = 1 # three set


r = cv.erode(im, el * 255, anchor=(1,0)) # anchor on the center (left 1)
s = cv.dilate(r, el * 255, anchor=(2,0)) # anchor on the right (right 1)

print(im)
print(r)
print(s) # half-element shifts cancel each other out

[[0 1 0 0 0]
 [0 1 1 0 0]
 [0 1 1 1 0]]

[[0 0 0 0 0]
 [0 1 0 0 0]
 [0 1 1 0 0]]

[[0 0 0 0 0]
 [0 1 1 0 0]
 [0 1 1 1 0]]
3

Hello,
I thank you a lot for taking the time to read me and answering me. However, I am sorry I should disagree with part of your answer…

First of all, the problem I am talking about has nothing to do about the shift of the anchor point… I understand this problem of anchor point, but it’s not the point here. In your answer, by changing the anchor point position, you actually have the same result than a flip… I can build a non symmetrical structuring element where changing the anchor point position will never produce the expected result.
Here is another example:

image = numpy.uint8([[0,0,0,0,0],[0,1,0,0,0],[0,1,1,0,0],[0,1,1,1,0],[0,0,0,0,0]])
el = numpy.uint8([[1,1,0],[0,1,1],[1,0,0]])
r = cv2.dilate(image, el)
print(image)
print("***")
print(r)

[[0 0 0 0 0]
 [0 1 0 0 0]
 [0 1 1 0 0]
 [0 1 1 1 0]
 [0 0 0 0 0]]
***
[[0 0 1 0 0]
 [1 1 1 1 0]
 [1 1 1 1 1]
 [1 1 1 1 0]
 [0 1 1 1 1]]

With Matlab (or other softwares), or when computing the dilation by hand according to its definition, the result is

1     1     0     0     0
1     1     1     0     0
1     1     1     1     0
1     1     1     1     1
1     1     1     0     0

The bottom right pixel, for example, should not be equal to 1 but to 0 in the result when considering the definition of the dilation. I don’t think that changing the anchor point can solve this problem (do you think it can?).

The flip operation must be done when doing the dilation, and not done for the erosion, because it is inherently defined like this! It is not a matter of taste, it is the definition of the dilation and the erosion, that you can find in books, on wikipedia, or elsewhere.

This flip operation is “vital” in order to produce a good result for the opening (erosion followed by a dilation) or closing. On OpenCV, this brings serious issues like morphological openings not producing anti extensive results, which they should always do (Opening (morphology) - Wikipedia). In other words, here, the result of the opening should be included in the original shape. Here is another example with the same structuring element but a different image:

image = numpy.uint8([[0,0,0,0,0],[0,1,1,0,0],[0,1,1,1,0],[0,1,1,1,0],[0,0,0,0,0]])
el = numpy.uint8([[1,1,0],[0,1,1],[1,0,0]])
r = cv2.morphologyEx(image, cv2.MORPH_OPEN, el)
print(image)
print("***")
print(r)

[[0 0 0 0 0]
 [0 1 1 0 0]
 [0 1 1 1 0]
 [0 1 1 1 0]
 [0 0 0 0 0]]
***
[[0 0 0 0 0]
 [0 0 0 1 0]
 [0 1 1 0 0]
 [0 0 1 1 0]
 [0 0 0 0 0]]

Morphological opening should only lower down the values of the original pixels, never raise them up. But here, because of the implementation problem of the dilation, the pixel located at the second row from the top and fourth column from the left, went from 0 to 1: it should never happen with the opening. As a result, here, the result of the opening is not included in the original shape… And we can’t perform any shifting of the anchor point to solve the issue.
And again, other softwares like Matlab or pymorph do not have this problem and produce the good result for the opening.
I think the problem outlined with the opening definitely proves that there is something wrong with OpenCV implementation of the dilation, as it should be an anti extensive transformation and it is not the case here. Moreover, why would the results of OpenCV and other softwares differ so much if there was no problem?

To sum up, here are the clues I have to say that morphological dilation is not well implemented in OpenCV:

  • Results of dilation differ in OpenCV compared to other softwares like Matlab and pymorph, or when doing the operation by hand following the definition in books.
  • Results of the opening operation in OpenCV is not anti extensive although it should be (and the result is anti extensive in Matlab and Pymorph)

This problem does not happen when considering symmetrical structuring elements, which are most of the time used in morphological operations. However, non symmetrical structuring elements should work also, and it is not the case…

Again, thanks for reading up to here !