calib3d probably depends on core and a few others of the core modules.
after you’ve generated (with cmake) some “build files” for OpenCV, you can find the calib3d target/project, build that. it should also start building dependencies. you can ignore all the other targets/projects.
since you’re talking about linkage, you would not be touching any Python bindings. those would have roped in all the modules that are enabled for building.