What error value did you get from the calibration? I suspect it was pretty high? (I’d say shoot for a value below 1.0 - if you are getting huge error values it tells you that the calibration didn’t model the data very well, which could lead to results like you are getting.)
The obvious issues I see with your calibration process:
- calibration target isn’t very flat. print it again and adhere it to a piece of glass or something else really flat. or at a minimum re-print it and avoid the wrinkles.
- you need to move the camera (or the target) in a way that gets more perspective distortion in at least some of the images. Most of your images look to be “straight on” shots - try some with 30 degree rotation between the calibration target and the image plane.
- you need more data from close to the corners of the image. Since you are using the Charuco target you don’t need to see the whole thing - position the camera / target so you get points as far into the corners as you can. This will greatly help with getting accurate distortion parameters.