that’s generally a bad idea, unless you feed that orientation into the calculations (I don’t know off-hand if OpenCV even allows that).
you need the pattern to be sitting obliquely. the picture must show foreshortening/perspective.
OpenCV’s intrinsic calibration gives you poses, not for the camera, but for the boards you view. you can ignore those. they are secondary.
if you’re planning to estimate distortion yourself, that’s a lot of extra work. I’d strongly recommend just using the existing calibration routine.