Android OpenCV 4.8 solvePnPRansac algorithm is failing

Android/Java
1

I have the following setup:

Android Studio Giraffe | 2022.3.1 Patch 1.
Runtime version: 17.0.6+0-17.0.6b829.9-10027231 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.

I am using an emulator using API Level 23. Android 6.0.

I am running a sample app of the opencv-4.8.0-android-sdk, in particular the image-manipulations sample. I have modified this app to test the PnP Ransac algorithm. The modifications I made are as follows. In the onResume method, I have put the following code that loads OpenCV library, set up a set of points, and run the solvePnPRansac algorithm:

OpenCVLoader.initDebug();     
    Mat mDistortionCoefficients = new Mat();
    Mat.zeros(5, 1, CvType.CV_64FC1).copyTo(mDistortionCoefficients);
    MatOfDouble distortion = new MatOfDouble(mDistortionCoefficients);

    /////////////////////////////
    // set up 3d points
    MatOfPoint3f points3d = new MatOfPoint3f();

    Point3[] point1 = new Point3[8];
    point1[0] = new Point3(-0.98432022, 7.0196648, 2.133873);
    point1[1] = new Point3(1.0676798, 7.0196815, 2.1338277);
    point1[2] = new Point3(-0.98432022, 7.0196643, 2.1338732);
    point1[3] = new Point3(1.0676798, 7.0196815, 2.1338279);
    point1[4] = new Point3(-0.98432022, 7.0196643, 2.1338735);
    point1[5] = new Point3(1.0676798, 7.0196815, 2.1338279);
    point1[6] = new Point3(-0.98432022, 7.0196643, 2.1338735);
    point1[7] = new Point3(1.0676798, 7.019681, 2.1338279);


    java.util.List<Point3> points3dList = new ArrayList<Point3>();
    for (int i = 0; i<8; i++) {
        points3dList.add(point1[i]);
    }

    MatOfPoint3f points3DMat = new MatOfPoint3f();
    points3DMat.fromList(points3dList);


    /////////////////////////
    // set up 2d points

    MatOfPoint2f points2dMat = new MatOfPoint2f();

    ArrayList<Point> lstPt = new ArrayList<Point>();
    lstPt.add(new Point(1029.3856, 433.59375));
    lstPt.add(new Point(970.5932, 609.9375));
    lstPt.add(new Point( 713.53815, 510.23438));
    lstPt.add(new Point(706.43353, 741.79688));
    lstPt.add(new Point(454.50211, 480.60938));
    lstPt.add(new Point(775.47668, 749.4375));
    lstPt.add(new Point(134.81992, 554.10938));
    lstPt.add(new Point(130.31779, 771.96094));

    points2dMat.fromList(lstPt);


    //////////////////////


    Mat cameraMat = new Mat();
    Mat.eye(3, 3, CvType.CV_64FC1).copyTo(cameraMat);
    cameraMat.put(0, 0, 640);
    cameraMat.put(1, 1, 711.6292648861155);
    cameraMat.put(0, 2, 640);
    cameraMat.put(1, 2, 480);


    Mat rVecs = new Mat();
    Mat tVecs = new Mat();

    Mat output = new Mat();

    boolean res = Calib3d.solvePnPRansac(points3DMat, points2dMat, cameraMat, distortion, rVecs, tVecs);
   
   
    Log.i("Tag", "rVecs:  " + rVecs.dump());
    Log.i("Tag", "tVecs:  " + tVecs.dump());

The solvePnPRansac algoirhtm is not running correctly. It basically returns “false” and the rVecs and tVecs are (0,0,0).

I have run this code in another app that runs the opencv 4.1.2 and the algorithm seemed to run successfully, and return sensible tVecs and rVecs. I was wondering what I could do in order to debug this futher and understand why the Ransac algorithm fails, or whether there is an incorrect setup?

If instead of the ransac algorithm I use the solvePnP, the algorithm succees:

    boolean res = Calib3d.solvePnP(points3DMat, points2dMat, cameraMat, distortion, rVecs, tVecs);

In this later case, the rVecs and tVecs obtained are sensible values…

rVecs: [-2.198081857322169;
-1.63624099961629;
0.3348899410673954]
tVecs: [-5.462323286986146;
2.492109946671164;
11.16714759567074]

I am unsure why ransac is not working properly. Any help would be appreciated.

2

that has got a few more parameters that are optional. play with them? perhaps the default values aren’t a good choice for your data?

3

Thanks. Will give it a go. I am just surprised observing different behaviour in different OpenCV versions…

4

if you literally see a newer version of opencv fail on the exact same code and data on which an earlier version succeeded (and should succeed), then that’s a regression. you could browse the issues on opencv’s github. maybe someone reported it already. maybe you’re the first.

5

Before reporting it, I will do a bit more testing to ensure (1) I put valid data, and (2) compare the same code with different versions.

6

I think the issue I had is the input data. The data that I was feeding might not have been good enough. I got new data and I got sensible results using the OpenCV 4.8 library. Maybe the algorithm in the 4.1 has different level of tolerance when deciging if the solvePnP success or fails…