Best practices for data collection: stereo camera calibration with wide baseline and fisheye lenses


I’m working on calibrating my stereo camera. It’s equipped with two 160degrees FOV cameras and has a 30cm baseline. As I’m struggling with the task and understood that good data collection is key, I hoped someone with calibration experience can share their insights into how to collect data properly, especially for a challenging task like fisheye stereo calibration.

I’m using a big charuco board for calibration: 170x110cm, with 17x11 checkers. I’ve taken images from different viewpoints around 3-4 metres away from the board. For single camera calibration I achieve reprojection errors of around 1 and 2 respectively, but the reprojection error for the stereo calibration is around 70.

Attached an example image from the dataset.

this pretty much sums it up: