How to apply feature matching method to enhance Stereo Disparity Map result

Python
, , , ,
1

Recently I’m using the Middlebury Stereo Datasets 2005 for generating the disparity map. Only the dataset Art, Dolls and Reindeer will be used. The requirement is to generate the disparity map with only view1.png and view5.png for each set.

I’ve tried to generate the depth map directly with both cv2.StereoSGBM and cv2.StereoBM which didn’t really give out a satisfying result, here is the result of Art with the StereoSGBM code:

imgL = cv2.imread(‘./Art/view1.png’, cv2.IMREAD_GRAYSCALE)
imgR = cv2.imread(‘./Art/view5.png’, cv2.IMREAD_GRAYSCALE)
win_size = 3
stereo = cv2.StereoSGBM_create(
minDisparity=20,
numDisparities=200,
blockSize=7,
uniquenessRatio=10,
speckleWindowSize=3,
speckleRange=1,
P1=8 * 3 * win_size ** 2,
P2=32 * 3 * win_size ** 2,
)
disparity_SGBM = stereo.compute(imgL, imgR)

disp1
disp11390×1110 149 KB

Other than just StereoSGBM and StereoBM, I also saw people using the feature matching method with ORB / SIFT for running cv2.warpPerspective before computing the depth map, however, the way it transforms seems to failed with a bad transformation

After generating depth maps with the above methods, I have also implemented the wls filter but I got confused with the Lamba and sigma value. I would like to ask how should I implement these method to enhance the output

2

what don’t you like about the output?

3

please show us, what you did there.

have you seen the tutorial ?

4

i would like to know if there is a right implementation for using ORB or SIFT w/ cv2.warpPerspective in order to get a better result, I’m using these code to get the psnr for comparing the original disp1 with the depth map I generated:

def psnr(img1, img2):
mse = numpy.mean( ((img1 - img2)) ** 2 )
if mse == 0:
return ‘INF’
PIXEL_MAX = 255.0
return 20 * math.log10(PIXEL_MAX / math.sqrt(mse))

test_imgs = [“Art”, “Dolls”, “Reindeer”]

for index in range(1):
gt_names = “./gt/”+test_imgs[index]+"/disp1.png";
gt_img = numpy.array(Image.open(gt_names),dtype=float);
print(gt_img.shape)

  pred_names =  "./pred/"+test_imgs[index]+"/disp1.png";
  pred_img = numpy.array(Image.open(pred_names),dtype=float);
  print(pred_img.shape)
    
  [h,l] = gt_img.shape
  gt_img = gt_img[:, 250:l]
  pred_img = pred_img[:, 250:l]
  pred_img[gt_img==0]= 0

  peaksnr = psnr(pred_img,gt_img);
  print('The Peak-SNR value is %0.4f \n', peaksnr);
5

sorry that i didnt show the result as im new to the forum and i couldnt upload more than i image:

win_size = 3
left_matcher = cv2.StereoSGBM_create(
minDisparity=20,
numDisparities=200,
blockSize=7,
uniquenessRatio=10,
speckleWindowSize=3,
speckleRange=1,
P1=8 * 3 * win_size ** 2,
P2=32 * 3 * win_size ** 2,
)

sigma = 2
lmbda = 600

right_matcher = cv2.ximgproc.createRightMatcher(left_matcher)
left_disp = left_matcher.compute(imgL_undistorted, imgR_undistorted)
right_disp = right_matcher.compute(imgR_undistorted,imgL_undistorted)

wls_filter = cv2.ximgproc.createDisparityWLSFilter(left_matcher)
wls_filter.setLambda(lmbda)
wls_filter.setSigmaColor(sigma)
filtered_disp = wls_filter.filter(left_disp, imgL, disparity_map_right=right_disp)

plt.imshow(filtered_disp, “gray”)

Figure_1