I know this topic is old and the issue has probably been resolved, but since I had the exact same problem, I thought I’d share the solution in case it helps someone else.
I had the exact same issue, and the T vector values were completely incorrect for me. For example, it showed a 17 cm distance along the Z-axis, while in reality, there was almost no distance at all.
I made two major changes when taking photos of the chessboard, and that resolved the issue.
First – Instead of using a printed chessboard, I used a tablet and displayed the chessboard pattern on its screen. I then took pictures of it simultaneously using two cameras. With the printed version, the edges of the paper were slightly warped, and there were some shadows on it—issues that were completely eliminated using the tablet.
Second – and probably the main reason – I took photos of the chessboard from different distances and angles. Previously, I used to place the chessboard flat on the ground (i.e., on an x-y plane), so the distance from the cameras was almost constant and rotation happened only within the x-y plane. But in the new method, I held the tablet at various angles and distances when taking the photos.
In the first method, I took 50 pairs of photos, but the calibration results were completely off. In the new method, I only took 14 photos, and the results were completely accurate.
There might have been other contributing factors as well, but whatever the case, this method solved the problem for me. I hope it helps solve yours too.