Bad Disparity map with SGBM algorithm

Giuliano_Cerchiara
Thu, 31 Mar 2022 21:21:01 +0000
Python

Good afternoon, I am looking to obtain a good disparity map with the SGBM algorithm to be able to approximate the depth of the objects in the image. The problem is that the disparity map gives me something wrong, since the closest objects appear to me in black, some in the middle in white and in the background black again.

my code to get the disparity map is as follows:

def depth_map(imgL, imgR):

    window_size = 2  # wsize default 3; 5; 7 for SGBM reduced size image; 15 for SGBM full size image (1300px and above); 5 Works nicely

    left_matcher = cv2.StereoSGBM_create(
        minDisparity=-1,
        numDisparities=16*16,  # max_disp has to be dividable by 16 f. E. HH 192, 256
        blockSize=window_size,
        P1=9 * 3 * window_size,
        # wsize default 3; 5; 7 for SGBM reduced size image; 15 for SGBM full size image (1300px and above); 5 Works nicely
        P2=128 * 3 * window_size,
        disp12MaxDiff=12,
        uniquenessRatio=40,
        speckleWindowSize=50,
        speckleRange=32,
        preFilterCap=63,
        mode=cv2.STEREO_SGBM_MODE_SGBM_3WAY
    )
    right_matcher = cv2.ximgproc.createRightMatcher(left_matcher)
    # FILTER Parameters
    lmbda = 70000
    sigma = 1.7
    visual_multiplier = 6

    wls_filter = cv2.ximgproc.createDisparityWLSFilter(matcher_left=left_matcher)
    wls_filter.setLambda(lmbda)
    wls_filter.setSigmaColor(sigma)

    displ = left_matcher.compute(imgL, imgR)  # .astype(np.float32)/16
    dispr = right_matcher.compute(imgR, imgL)  # .astype(np.float32)/16
    displ = np.int16(displ)
    dispr = np.int16(dispr)
    filteredImg = wls_filter.filter(displ, imgL, None, dispr)  # important to put "imgL" here!!!
    filteredImg = cv2.normalize(src=filteredImg, dst=filteredImg, beta=0, alpha=255, norm_type=cv2.NORM_MINMAX);
    filteredImg = np.uint8(filteredImg)

    return filteredImg

I pass my 2 rectified images to it, and it returns the normalized disparity map so I can see it.

The problem is that, as can be seen in the following links, my disparity map is wrong. You can see the objects in the image, but something is missing.

The characteristics of my equipment are the following:

2 ELP 13MP cameras without distortion, the resolution of each image I take is (1944,2592).

Distance between cameras (baseline) = 145 mm

The code with which I capture the frames, dedicates a thread to each camera for the capture of the frames (so I get the images with a difference <11ms)

The cameras are fixed to a 3d print with screws, to make sure they don’t move after the stereo calibration.

First, each camera was individually calibrated with Charucoboard, obtaining a reprojection error <0.25 in both cameras.

Then, they were calibrated in Stereo, they were calibrated with Chessboard, obtaining a reprojection error of 0.179.

Attached in the following:

Disparity Image Normalized

Undistorted Left Image

Undistorted Right Image

I tried to tweak each parameter inside the SGBM function, but to a greater or lesser extent it stays the same. I have managed to get the white color to appear in the front of the image, with “numdisparity=2*16”, but even so there is a lot of mixing of colors, since the objects are distant each other. If anyone knows what I’m doing wrong, I’d really appreciate it!

14 posts - 4 participants