# Calibrate from Unity3D model data **URL:** \ **Category:** Python\ **Tags:** calib3d\ **Created:** [February 24, 2023, 12:49pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112 "2023-02-24T12:49:40Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![TsingLoo](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TsingLoo](https://forum.opencv.org/u/TsingLoo)\ **Post date:** [February 24, 2023, 12:49pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/1 "2023-02-24T12:49:41Z")
Hi everyone, I am new to here and I get some troubles with calibration. I am going to synthesize frames in Unity3D as dataset(Like [MultiviewX](https://github.com/hou-yz/MultiviewX) ) for further CNN trainning. For the first step, I need to calibrate the camera to get the intrinsics and extrinsics of the camera. Yes, there are some mathmatcial methods to get the intrinsics and the extrinsics. However, it could be better if I can get the result from OpenCV. To finish the calibration, I setup the scene with many cubes(like figure below). ![0000](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/a/a6b33051bc203d7f81dc8f1c1c4d51d016650621.jpeg) Then, the world and screen space coordinates of the cubes could be retrieved easily by gameobject.transform.position and Camera.WorldToScreenPoint(). Finally, I get pairs of points from world and screen. In python, I use `cameraMatrix = cv2.initCameraMatrix2D(points_3d, points_2d, (IMAGE_HEIGHT, IMAGE_WIDTH))` and ` retval, cameraMatrix, distCoeffs, rvecs, tvecs = \ cv2.calibrateCamera(points_3d, points_2d, (IMAGE_HEIGHT, IMAGE_WIDTH), cameraMatrix, None, flags=cv2.CALIB_USE_INTRINSIC_GUESS, )` They take in the 3d world and 2d screen corresponding points to do the calibration. Theoretically speaking, the 6 static same camera in the scene should get the same(at least close) intrinsics. Horever, in my case, only the first 4 of them could receive a good calibration` 9.3530752008341096e+02 0. 9.6000011291213366e+02 0. 9.3530728269909184e+02 5.4000007545182882e+02 0. 0. 1.`. The remainig 2 cameras failed ` 1.9522217456988033e+03 0. 5.3950000000000000e+02 0. 1.9522217456988033e+03 9.5950000000000000e+02 0. 0. 1`. I am confused about this, why is there the discrepancy? ---
**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\ **Post date:** [February 24, 2023, 1:07pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/2 "2023-02-24T13:07:09Z")
look at the cx, cy what do you notice? > [@TsingLoo](#): > > 9.6000011291213366e+02 > 5.4000007545182882e+02 > [@TsingLoo](#): > > 5.3950000000000000e+02 > 9.5950000000000000e+02 ---
**Author:** ![TsingLoo](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TsingLoo](https://forum.opencv.org/u/TsingLoo)\ **Post date:** [February 24, 2023, 2:00pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/3 "2023-02-24T14:00:55Z")
Thanks for your reply. It seems something get X,Y reversed. The distribution of the failed camera follows some rules, the bottom quarter: ![image](https://us1.discourse-cdn.com/flex020/uploads/opencv/original/2X/3/382f714af7e3245c515e7f4b97114bd47cd78386.jpeg) ---
**Author:** ![crackwitz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.opencv.org/crackwitz/32/14_2.png) [@crackwitz](https://forum.opencv.org/u/crackwitz)\ **Post date:** [February 24, 2023, 2:45pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/4 "2023-02-24T14:45:56Z")
maybe the intrinsic guess was initialized wrong. maybe it’s having trouble dealing with non-planar models (usually it’s checkerboards), or requires an additional flag for that. or you give it points that shouldn’t be in view, or are behind the camera? or something else is going on. in principle, a single “view” with enough points should be enough for calibration. the shadows of all those boxes don’t make neat lines on the ground. why’s that? just an optical illusion, different rows making shadows _almost_ in line with each other? ---
**Author:** ![TsingLoo](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TsingLoo](https://forum.opencv.org/u/TsingLoo)\ **Post date:** [February 24, 2023, 3:33pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/5 "2023-02-24T15:33:58Z")
Thanks for your reply. I think the initialized guess is wrong. I will upload the data tomorrow for further check. ---
**Author:** ![TsingLoo](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TsingLoo](https://forum.opencv.org/u/TsingLoo)\ **Post date:** [February 25, 2023, 1:11pm UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/6 "2023-02-25T13:11:25Z")
Here is the corresponding data: The 2D points and 3D points of each camera are saved in the `Cali` folder The calibration result is saved in the `calibration` folder By running the python script calibrateByCali.py, the calibration result will be generated. > **[GitHub - TsingLoo/MultiviewX\_FYP: Toolkits for the synthetic multiview...](https://github.com/TsingLoo/MultiviewX_FYP)** > > Toolkits for the synthetic multiview pedestrian detection dataset, MultiviewX. For XJTLU Year4 FYP - GitHub - TsingLoo/MultiviewX\_FYP: Toolkits for the synthetic multiview pedestrian detection data... ---
**Author:** ![TsingLoo](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TsingLoo](https://forum.opencv.org/u/TsingLoo)\ **Post date:** [March 3, 2023, 6:32am UTC](https://forum.opencv.org/t/calibrate-from-unity3d-model-data/12112/7 "2023-03-03T06:32:45Z")
It seems that the single view can not accomplish the calibration… I finally get the correct calibration by using this: [CalibrateTool](http://www.tsingloo.com/2023/03/01/0a2bf39019914a06954a4506b9f0ca37/) , which generates visual chessborad in different views.