Opencv VideoCapture is slow

I have a USB camera connected to Nvidia Jetson TX2. The camera is capable to deliver at 30fps. I have tried two software to capture video stream from this camera using c++, libUVC and OpenCV. I found that libUVC will deliver call back with YUYV image at 30 fps. However, opencv can achieve only 12 fps. Below is my opencv code:

cv::VideoCapture cap;
    cap.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('Y', 'U', 'Y', 'V'));
    cap.set(cv::CAP_PROP_FRAME_WIDTH, 640);
    cap.set(cv::CAP_PROP_FRAME_HEIGHT, 480);
    cap.set(cv::CAP_PROP_FPS, 30);

    // open the default camera, use something different from 0 otherwise;
    // Check VideoCapture documentation.
    if (!cap.open(0)) {
        ROS_ERROR("Can not open USB 8M");
        return false;
    }

    int count = 0;
    for (;;) {

        ROS_INFO("Get a frame");

        cv::Mat frame;
        cap >> frame;
        if (frame.empty()) break; // end of video stream
 }

Why is opencv so slow? Is there any problem with my code?