How do I get real time performance out of this CSRT object tracker?

1

Currently I’m getting less than 1fps using CSRT tracker on my Jetson Nano B01 with this code using opencv 4.10.0. 30 fps or more is what I’m shooting for. How can I change this code to improve the performance? I was getting 25-30 fps with this code on opencv 4.5.0. Could it be a version change that’s the difference? Or is there something I’m missing here?

void track_target(void)
{
/* Don't wrap the image from jetson inference until a valid image has been received.
   That way we know the memory has been allocaed and is ready. */
    if (valid_image_rcvd && !initialized_cv_image)
    {
        image_cv_wrapped = cv::Mat(input_video_height, input_video_width, CV_8UC3, image); // Directly wrap uchar3*
        initialized_cv_image = true;
    }
    else if (valid_image_rcvd && initialized_cv_image)
    {
        if (target_valid && !initialized_tracker)
        {
            target_bounding_box = cv::Rect(target_left, target_top, target_width, target_height);
            tracker_init(target_tracker, image_cv_wrapped, target_bounding_box);
            initialized_tracker = true;
        }

        if (initialized_tracker)
        {
            target_tracked = tracker_update(target_tracker, image_cv_wrapped, target_bounding_box);
        }

        if (target_tracked)
        {
            std::cout << "Tracking" << std::endl;
            cv::rectangle(image_cv_wrapped, target_bounding_box, cv::Scalar(255, 0, 0));

            tracking = true;
        }
        else
        {
            std::cout << "Not Tracking" << std::endl;
            initialized_tracker = false;
            tracking = false;
        }
    }
}
2

25-30 FPS in v4.5.0 and 1 FPS in 4.10.0 with the exact same code, camera/image size and format?

The things I would be looking for:

  1. Was the 4.10.0 version built the same as 4.5.0? The obvious thing to look for is debug vs release build - OpenCV in debug mode can be very significantly slower - 1fps vs 30fps isn’t out of the question.
  2. Is there anything that has changed with your camera / image source? The obvious thing is image size - but a factor of 30 slowdown is hard to imagine. Any chance your image source is providing images at a slower rate? 1 FPS frame rate would do it. :slight_smile:
  3. Are you running it on the same data? I don’t know how the tracker works, but I could imagine that runtime performance could vary significantly with the bounding box size, for example.

Have you tried switching back to 4.5.0 to confirm the performance? Is there a specific reason you want to use 4.10.0?

1 Like