Hello, I would be very thankful for any advice regarding openCV hand-eye calibration.
I am working on robotic station, where I need to pick planary objects using 7-DOF uFactory Robot
Robot is equipped with custom gripper and basler ace camera working of resolution 2046x2046.
The TCP of robot is set to middle of end-effector flange and pointing down.
The camera is mounted with approx. offsets:
x=-70 mm
y=0 mm
z=+60 mm
I have tried both charuco and classic chessboard to calibrate both the camera and the hand-eye using different scripts.
Both come with very similar results. The calibration of camera seems pretty straightforward, but there is one thing I am unsure:
Camera matrix result:
array([[4923., 0., 1016.],
[ 0., 4935., 1065.],
[ 0., 0., 1.]])
Isn’t it strange, that principal point (1016, 1065) is so far from center (1023, 1023)?
I don’t know if this affects following hand-eye calibration.
- Using different scripts the results seems promising, for example PARK
Translations:
array([[-69.87033895],
[ -2.82522451],
[ 65.0670899 ]])
Rotations:
array([[ 0.05361909, 0.99856145, 0.00014278],
[-0.99842409, 0.05360934, 0.01659471],
[ 0.01656318, -0.00103234, 0.99986229]])
Problem is, when I try to use the hand-eye results to move on specific marker, the resulting position is very inaccurate and there is
obvious problem mainly in Y axis calculation. My steps to verify:
- one aruco marker is scanned with camera/gripper pointing downwards aligned with Z axis
Z=650, Y=0, X=400, roll=-180, pitch=0, yaw=0the result position is [389.91190695 -31.11663879 -2.33965417]
- changing only Z axis to 550
the result is [390.43870439 -26.6417014 -2.39286384]
So only by changing the Z position of camera, the computed object position on Y axis is shifted 5 mm on static marker.
I also tried to use flag cv2.CALIB_FIX_PRINCIPAL_POINT when calculating camera matrix, which slightly changes the computed results, but the error trend on Y axis is still there.
My simplified code to use calibration:
#actual gripper position
x, y, z, roll, pitch, yaw = robotPose # [350, 0, 550, -np.pi, 0, 0]
#center of aruco marker
image_coords = np.array([[int(centerX)], [int(centerY)], [1]])
#z distance in camera frame is distance of gripper above base plane minus eye-hand z offset, right?
#-3 is because, the paper with marker is 3 mm under base plane
object_in_camera_coords = ((z-3)-hand_eye_translations[selectedMethod][2]) * np.dot(camera_matrix_inv, image_coords)
object_in_camera_coords = object_in_camera_coords.reshape((3,1))
object_in_gripper_coords = np.dot(hand_eye_rotation, object_in_camera_coords) + hand_eye_translation
robotMatrix = create_transformation_matrix(x, y, z, roll, pitch, yaw)
robot_to_base_rotation = robotMatrix[0:3, 0:3] #np.array([[1,0,0], [0,-1,0], [0,0,-1]])
robot_to_base_translation = robotMatrix[0:3, 3].reshape(3, 1)
object_in_base_coords = np.dot(robot_to_base_rotation, object_in_gripper_coords) + robot_to_base_translation
object_in_base_coords = object_in_base_coords.flatten()
More code:
def create_transformation_matrix(x, y, z, roll, pitch, yaw, degrees=False):
R = Rotation.from_euler('xyz', [roll, pitch, yaw], degrees=degrees).as_matrix()
t = np.array([x, y, z]).reshape(3, 1)
T = np.eye(4)
T[:3, :3] = R
T[:3, 3] = t.flatten()
return T
Images and robot poses used for calibration: HandEyeCalibrationData - Google Drive
Robot poses are pickled arrays [x,y,z,roll,pitch,yaw]
I really don’t know where to search for the problem, because thu numbers somehow corresponds with reality, but not accurately?
