How to measure FPS of an OpenCV camera preview?

Background
I’m currently working on an OpenCV camera application using Java (on Android Studio). The device that this application is launched on is a OnePlus Six. The application’s objective is to process each frame for object recognition.

The entire application as a whole is built around code within 3 main classes; MainActivity.java, activity_main.xml, AndroidManifest.xml. Thus, upon launching it, it jumps the user right into the camera preview in which the aforementioned activity is carried out.

However, I’m currently stuck at a part in which I’m attempting to print the framerate in the Android Studio console. I’m unclear as to how I can print the FPS within the Android Studio console. Any guidance would be greatly appreciated.

I had attempted to measure the applications framerate by inputting this into the code, to no avail.

public static void main(String [] args)
{
      get_fps();
}

double get_fps(double old_time) {
        double new_time = System.nanoTime();
        double delta = new_time - old_time;
        double fps = 1 / (delta * 1000);

        Log.i(TAG, "Estimated fps" + fps);

        return fps;
    }

I am attempting to print it so it can show up in the “RUN” tab (the console), but unfortunately, this method was unable to do that. However any other method to measure the FPS will work!

please explain the parts of that code, i.e. what every line does, and why you think that needs to be done.

I should probably have included the fact that I had gotten this code from this answer on Stack OverFlow. Essentially, the code is attempting to put into practice the action of attempting to count the frames its done since time elapsed. I’m also fairly new to programming, however, I do understand that the aforementioned answer is its intention

However, I do not want to restrict my path to JUST this code. If there’s another way of measuring the FPS, I would gladly use that instead.

yes, that’s a vague description and that’s the reason you couldn’t make use of that code in your situation.

I would recommend a general programming forum.

or you could anticipate needing to show your code (all relevant code) in order for people to give advice on it.

1 Like

That’s a great suggestion, seeing as though I’m unable to edit my original post to implement the necessary relevant code, I will paste it here in this comment. In developing this application, I had used this OpenCV tutorial in guiding myself.

MainActivity.java

public class MainActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2 {

 //initialize OpenCV manager.
 private BaseLoaderCallback baseLoaderCallback = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
                case LoaderCallbackInterface.SUCCESS: {
                    Log.i(TAG, "OpenCV loaded successfully");
                    mOpenCvCameraView.enableView();
                    break;
                }
                default: {
                    super. onManagerConnected(status);
                    break;
                }
            }
        }
    };

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        //Set up a camera listener.
        mOpenCvCameraView = (CameraBridgeViewBase)findViewById(R.id.CameraView);
        mOpenCvCameraView.setVisibility(CameraBridgeViewBase.VISIBLE);
        mOpenCvCameraView.setCvCameraViewListener(this);

        //Checking if the permission has already been granted
        if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
                == PackageManager.PERMISSION_GRANTED) {
            Log.d(TAG, "Permissions granted");
            initializeCamera((JavaCameraView) mOpenCvCameraView, activeCamera);
        } else {
            //prompt system dialogue
            Log.d(TAG, "Permission prompt");
            ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.CAMERA}, MY_CAMERA_REQUEST_CODE);
        }
    }

    //Load a network.
    @Override
    public void onCameraViewStarted(int width, int height) {
        String proto = getPath("MobileNetSSD_deploy.prototxt", this);
        String weights = getPath("MobileNetSSD_deploy.caffemodel", this);
        net = Dnn.readNetFromCaffe(proto, weights);
        Log.i(TAG, "Network loaded successfully");

    }

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        //return null;
        final int IN_WIDTH = 300;
        final int IN_HEIGHT = 300;
        final float WH_RATIO = (float)IN_WIDTH / IN_HEIGHT;
        final double IN_SCALE_FACTOR = 0.007843;
        final double MEAN_VAL = 127.5;
        final double THRESHOLD = 0.2;

        //Get a new frame
        Mat frame = inputFrame.rgba();
        Imgproc.cvtColor(frame, frame, Imgproc.COLOR_RGBA2RGB);

        //Forward image through network.
        Mat blob = Dnn.blobFromImage(frame, IN_SCALE_FACTOR,
                new Size(IN_WIDTH, IN_HEIGHT),
                new Scalar(MEAN_VAL, MEAN_VAL, MEAN_VAL), false, false);
        net.setInput(blob);
        Mat detections = net.forward();

        int cols = frame.cols();
        int rows = frame.rows();

        detections = detections.reshape(1, (int)detections.total() / 7);

        for (int i = 0; i < detections.rows(); ++i) {
            double confidence = detections.get(i,2)[0];
            if (confidence > THRESHOLD) {
                int classId = (int)detections.get(i,1)[0];

                int left   = (int)(detections.get(i,3)[0] * cols);
                int top    = (int)(detections.get(i,4)[0] * rows);
                int right  = (int)(detections.get(i,5)[0] * cols);
                int bottom = (int)(detections.get(i,6)[0] * rows);

                //Draw rectangle around detected object.
                Imgproc.rectangle(frame, new Point(left,top), new Point(right,bottom),
                        new Scalar(0, 255, 0));
                String label = classNames[classId] + ": " + confidence;
                int[] baseLine = new int[1];
                Size labelSize = Imgproc.getTextSize(label, Imgproc.FONT_HERSHEY_SIMPLEX, 0.5, 1, baseLine);

                //Draw background for label.
                Imgproc.rectangle(frame, new Point(left, top - labelSize.height),
                        new Point(left + labelSize.width, top + baseLine[0]),
                        new Scalar(255, 255, 255), Imgproc.FILLED);
                //Write class name and confidence.
                Imgproc.putText(frame, label, new Point(left, top),
                        Imgproc.FONT_HERSHEY_SIMPLEX, 0.5, new Scalar(0, 0, 0));
            }
        }
        return frame;
    }

    private void initializeCamera(JavaCameraView mOpenCvCameraView, int activeCamera){
        mOpenCvCameraView.setCameraPermissionGranted();
        mOpenCvCameraView.setCameraIndex(activeCamera);
        mOpenCvCameraView.setVisibility(CameraBridgeViewBase.VISIBLE);
        mOpenCvCameraView.setCvCameraViewListener(this);
    }

you’ll need to do your FPS calculations inside of onCameraFrame.

as alluded to in that get_fps snippet, you’ll need to figure out the time difference since the last frame. you will need to keep that timestamp and update it every time.

Okay, wow that makes sense! It’s only appropriate for me to move my framerate measurement into the method that is responsible for processing the frames.

I’ve ended up moving the get_fps snippet into OnCameraFrame and double-slashed the snippet being called in the main method. However, I’ve encountered several errors here. The main ones being one you had pointed out in your previous response; old_time requires a value input because there isn’t anything assigned to it, which is also a concern I should have included in OP as it was an issue I had prior to posting. Also, the variables get_fps are never used, so would I have to use it sometime within the OnCameraFrame method?

Aside from that, these seem to be the errors I have faced and have resolved everything else.

Alright all, so I’ve worked on a new piece of code that was originally written in Python, but i had translated to Java. So, the code shows up with zero errors, however… the framerate has still not been printed! I’ve been traversing the internet for a reason as to why this is the case. Could anybody explain to me as to why

import org.opencv.core.Mat;
import org.opencv.videoio.VideoCapture;

import java.time.OffsetDateTime;
import java.time.temporal.ChronoUnit;

    public static void main (String[] args) {
        //Pass your video capture argument
        VideoCapture cap = new VideoCapture(0);
        printFps(cap);
    }

    // Takes an OpenCV VideoCapture as an argument.
    public static void printFps(VideoCapture capture) {
        // Completely arbitrary number of frames but 120 is a decent sample.
        int num_frames = 120;

        // Declare variables for timekeeping.
        // time1 = where the sample start time will be stored
        // time2 = where the sample end time will be stored
        OffsetDateTime time1, time2;

        // Create a new Matrix (OpenCV's variable for storing a video frame)
        Mat frame = new Mat();

        // Print out a message indicating frames are
        // starting to be captured now.
        System.out.println("Capturing frames now.");

        // Save the current time in the first variable.
        time1 = OffsetDateTime.now();

        // Capture a number of frames
        for (int index = 0; index < num_frames; ++index) {
            // Store the frame in the frame variable
            capture.read(frame);
        }

        // When this operation finishes, store the current time in the second variable.
        time2 = OffsetDateTime.now();

        // Calculate the time that passed during the captures
        long secondsBetween = ChronoUnit.SECONDS.between(time2, time1);

        // Calculate frames per second
        int fps = num_frames / (int) secondsBetween;
        Log.i(TAG, "Estimated FPS: " + fps);

        // Release the VideoCapture object
        capture.release();
    }

EDIT: added code to call the printFps function

is anything calling that function?

My goodness, yes! There is something that had called that function that I had forgotten to add. Let me just add it into the previous post. But for reference, it was being called by

    public static void main (String[] args) {
        //Pass your video capture argument
        VideoCapture cap = new VideoCapture(0);
        printFps(cap);
    }

But the point still stands, the FPS has not been printed within the RUN tab of the Android Studio console.

If Log.i() doesn’t work, that’s really not an OpenCV issue anymore… You could use other means to find out what the value is.

1 Like

@matti.vuori Would you be able to expand on what “other means” I could use?

that code calls that function once.

it should now read frames from the capture object and then print something… hopefully.