I found calibrateCamera to converge more reliably than the fisheye counterpart, but the fisheye calibration was much better at extrapolation.
My solution with a similar FOV lens was to use the Charuco calibration target to get more data closer to the edges/corner of the image. I found the rational model to work very well when you have sufficient edge/corner data.
As crackwitz points out, the quality of the data is critical. In my case I had to extend the calibration process to iteratively find more calibration target points. Basically:
- Run the charuco calibration to get an initial result.
- Use the current calibration result to predict the image location for other nearby chessboard corners (that is, ones that were not included in the current calibration, but are adjacent to ones that were used)
- Use goodFeaturesToTrack / cornerSubPix to get a precise image position for the new points.
- Add these new points to your calibration data and re-run calibration.
- Repeat 2-4 until some stopping criteria is met
This might not normally be necessary, but I found that with high distortion the normal algorithm would fail to find points as you got closer to the edge. It seems like the estimate of where the chessboard corner is (based on the adjacent ArUco marker locations) wasn’t working well with high radial and perspective distortion, but I’m not sure.