I have been working on an object detection problem in opencv on Android using a c++ library inside of Unity. All of my opencv code is in c++ and I’m using version 4.4.0. I have calibrated my camera and I know the camera matrix and distortion parameters.
I use a custom tflite library to detect a series of points on my real world object, then I use solvePnP to find the rotation and translation for the 3D model of that object. In opencv land it all works really well. I can use solvepnp to get the rotation vector, rotation matrix, and translation vector. Then I can use projectPoints to reproject the points and when I do they line up more or less with the detected points (with a RMSE of 50 or less, usually around 26).
My trouble is in translating this success to Unity. I have taken the rotation vector (tried the matrix too), and the translation vector and tried to use it to translate my 3D model in Unity into the position. I cannot seem to get it right 
It always seems to be off by some combination of 180,90 or 45 degrees. It’s never off wildly so I suspect I have not translated something properly.
To start off I have a model. I got the model points by placing my model in unity at 0,0,0 and exporting the points we will be detecting. Now Unity has a coordinate system like openCV except the y axis points up (right handed). I do not do anything to change the model into openCV (left handed) but I did once try negating the y values which did not help.
Next I detect the points and make sure they are lined up with the right model points and I run
cv::solvePnP(model_points, image_points, camera_matrix, dist_coeffs, rotation_vector, translation_vector, false, cv::SOLVEPNP_ITERATIVE );
I get the rotation matrix in case I need it
cv::Mat rotation_matrix(3,3,cv::DataType::type);
cv::Rodrigues(rotation_vector, rotation_matrix);
Next I reproject my points. I will see the found points, and reprojected points on the camera frame up in Unity. They will align very closely.
cv::projectPoints(model_points, rotation_vector, translation_vector, camera_matrix, dist_coeffs, projected_points.points);
I calculate the RMSE and check if it is 50 or less. If so I consider this a valid rotation and translation and I pass it to Unity.
Now I know Unity has a coordinate system difference so I will try to account for that. I have read everything I can find on people who have come before me But I am still failing. Anyway My model starts at 0,0,0. My translation vector from opencv is passed in as the first three floats in an array. I negate the y axis and update the objects position. This part actually seems to work fine.
//yaxis is inverted
transform.position = new Vector3(t[0], -t[1], t[2]);
Now for the rotation. I have done this with both rotation vector and matrix and the results are the same. For rotation vector I understand that it is a vector showing the axis of rotation and its magnitude is the angle of rotation. So I calculate the magnitude in radians:
float theta = (float)(Math.Sqrt(x * x + y * y + z * z) * 180 / Math.PI);
then I negate the x and z components of the axis:
Vector3 axis = new Vector3(-x, y, -z);
Finally I assign the new rotation to the local rotation of my model.
Quaternion rot = Quaternion.AngleAxis(-theta, axis);
transform.localRotation = rot;
I’m always wrong though. I’ve also tried negative theta which does not work either. However it always seems to be off by 45,90,180 degrees. For example in one camera position I can fix it like this:
transform.localEulerAngles.x - 90f, transform.localEulerAngles.y + 180f,
However when I move my camera to another angle it is wrong again. Anyway I’m stuck so I thought I would ask if anyone had any ideas what I might be doing wrong.
Thanks for any advice you can give.
