Panorama stitching using OpenCV in Python

Python
,
1

Given 18 quadratic cubemaps (aspect ratio 1:1, resolution 1000x1000) with a FOV of 90°, I’m trying to convert them into an equirectangular panorama (aspect ratio 2:1, resolution 4000x2000). The images overlap a lot. My first approach uses the Stitcher class and the following Python code.

import cv2 as cv
import glob

images = []
file_paths = glob.glob('*.png')

for file_path in file_paths:
    images.append(cv.imread(file_path, cv.IMREAD_COLOR))

stitcher = cv.Stitcher.create(cv.Stitcher_PANORAMA)
status, pano = stitcher.stitch(images)

if status != cv.Stitcher_OK:
    print('ERROR {0}: The images could not be stitched.'.format(status))
    exit()

cv.imwrite('panorama.png', pano)

This approach will either fail with error code 3 (ERR_CAMERA_PARAMS_ADJUST_FAIL) or it will produce artifacts and only half of the panorama will be stitched. Unfortunately, I have not been able to find out under which conditions the approach will fail completely and when it will at least produce a result.

panorama_stitched_cv
panorama_stitched_cv1000×500 44.3 KB

I have already tried to change the order of the images, but this only results in different artifacts or the stitching fails completely with error code 3 (ERR_CAMERA_PARAMS_ADJUST_FAIL). Then I have tried to adjust the warper which does not seem to be possible in Python. Finally, I have tried to stitch the images manually, but the cv::detail::FeaturesFinder is not available in Python. The discontinued Microsoft Image Composite Editor can be used to successfully stitch the panorama without artifacts, but the stitching process cannot be automated (using Python).

2

Another approach would be to stitch only six cubemaps together, since each pixel can be uniquely assigned to a cubemap.

panorama_stitched_numpy
panorama_stitched_numpy1000×500 68.6 KB

The only problem is that the lighting is different in each cubemap. I tried using the MultiBandBlender class to blend the edges, but I can’t get it to work properly. The function Panorama.load loads the image using imread and Panorama.cubes_to_panorama_gpu converts the six images to the panorama and returns the masks for each cubemap.

images = []
images.append(Panorama.load('N.png'))
images.append(Panorama.load('S.png'))
images.append(Panorama.load('E.png'))
images.append(Panorama.load('W.png'))
images.append(Panorama.load('U.png'))
images.append(Panorama.load('D.png'))

pano, faces = Panorama.cubes_to_panorama_gpu(images)

blender = cv.detail.MultiBandBlender(try_gpu=1, num_bands=5)
blender.prepare((0, 0, pano.shape[1], pano.shape[0]))

for face, image in enumerate(images):

    mask = np.zeros(faces.shape, dtype=np.uint8)
    mask[faces == face] = 255

    blender.feed(pano, mask, (0, 0))

result, _ = blender.blend(None, None)

Panorama.save('panorama.png', result)
3

related: