Wrong rotations on chessboard

Python
1

Hi everyone,
i have been having problem with the chessboard pose identification, using a method i have always used.
I have a set of chessboard images taken with a high res cam(5000x5000).
My pipeline works fine to detect the chess, i downsize the image(findchessboard wont work on this huge images), i findchessboard, cornerSubPix, and solvePnp.
The problem is that the orientation of the frame on the chessboard is not coherent in all the images.
As a reference i upload an image in which in wihch the frame is wrongly posed:

pd_5
pd_51920×1920 63.3 KB

The code i use is the following :
def get_chess(file_in,cnt):
print(“getting image”)
HS = 8
WS = 9
size = (WS, HS)
square_pix = 0.02
matPoint = []

# for i in range(HS):
#     for j in range(WS):    
#         matPoint.append((float(i)*square_pix - ((float(HS-1)/2.0)*square_pix),
#                             float(j)*square_pix - ((float(WS-1)/2.0)*square_pix),0.0))
# for i in range(HS):
#     for j in range(WS):    
#         matPoint.append((float(i)*square_pix, float(j)*square_pix ,0.0))
# matPoint_np = np.asarray(matPoint)

matPoint_np = np.zeros((8*9,3), np.float32)
matPoint_np[:,:2] = np.mgrid[0:9,0:8].T.reshape(-1,2) * square_pix

camera_matrix = np.array( [ 4.8595692824480102e+03, 0.                    , 2.5642702555935216e+03, 0., 4.8595692824480102e+03, 2.5688489623880660e+03, 0., 0., 1. ] ).reshape((3, 3))
camera_dist   = np.array( [ -9.265718581217025e-02, 8.2862953714419874e-02, 0.                    , 0., 0. ] )
img_in = cv.imread(file_in)
gray = cv.cvtColor(img_in, cv.COLOR_BGR2GRAY)
resized = resize_img(gray)

status,corners = cv.findChessboardCorners(resized, size, None)

if status == True:
    scale_percent = 20 
    corners = corners * int(100 / scale_percent)
    criteria = (cv.TERM_CRITERIA_EPS + cv.TermCriteria_COUNT, 40, 0.001)
    corners2 = cv.cornerSubPix(gray, corners, (17,17), (5,5), criteria)
    ret,rvecs, tvecs = cv.solvePnP(matPoint_np, corners2, camera_matrix, camera_dist)
    print(cnt,rvecs)
    axis = np.float32([[1,0,0], [0,1,0], [0,0,1]]).reshape(-1,3)
    imgpts, jac = cv.projectPoints(axis, rvecs, tvecs, camera_matrix, np.asarray([0.,0.,0.,0.,0.]))
    corner = tuple(corners2[0].ravel())
    corner = tuple(int(tup) for tup in corner)
    p1 = tuple(int(tup) for tup in imgpts[0].ravel())
    p2 = tuple(int(tup) for tup in imgpts[1].ravel())
    p3 = tuple(int(tup) for tup in imgpts[2].ravel())
    img = cv.line(gray, corner, p1, (255,0,0), 3)
    img = cv.line(gray, corner, p2, (0,255,0), 21)
    img = cv.line(gray, corner, p3, (0,0,255), 41)
    img = cv.drawChessboardCorners(gray, size, corners2, True)
    # cv.imshow('img',img)
    cv.imwrite("/tmp/pd_" + str(cnt) + ".png",gray)
    # k = cv.waitKey(0) & 0xFF

    dst, jacobian = cv.Rodrigues(rvecs)
    new_t = np.array([[dst[0,0], dst[0,1], dst[0,2], tvecs[0,0]],
                      [dst[1,0], dst[1,1], dst[1,2], tvecs[1,0]],
                      [dst[2,0], dst[2,1], dst[2,2], tvecs[2,0]],
                      [      0,        0,        0,          1]],dtype=np.float64)
    return new_t

Can someone help me get what is going wrong?
Thanks

2

First one, I think you need to check the corner’s direction and fix it. Second, Your Pose will be affected when you define the word coordinate system. So you need to check the coordinate system is correct. Hope the message is useful to you.