I can’t predict that, don’t have experience with such setups.
imagine the epipolar lines. calibration would probably have to severely contort the views.
the involved implementations make some assumptions. I am not sure what you are doing falls into those assumptions.
you’d best grab all the books on this that you can, and do literature research for papers.