Camera calibration algorithm returns large standard deviation values for intrinsic camera parameters (fx, fy, cx, cy) and large overall reprojection error

I verified and indeed the corner finding algorithm returns some the set of corners both row wise and column wise. However, after filtering and making sure that all of the pairs have the same order the resulting RMS is worse than initially (increase in value of about ~0.3 pixels). Why would that be?