Operators are not executed by separated threads for GAPI streaming mode?

C++
1

Hi,
I am trying to use GAPI streaming mode to handle multi-source video streaming.
I expect that each stage(operator) can be executed by a separated thread.
Can anyone help correct the codes below?
Many thanks

2

int main(int argc, char *argv[])
{
cv::CommandLineParser cmd(argc, argv, keys);
cmd.about(about);
if (cmd.has(“help”)) {
cmd.printMessage();
return 0;
}
const auto input_file_name = cmd.getstd::string(“input”);
const auto streaming_queue_capacity = cmd.get(“queue_capacity”);
std::cout << "streaming_queue_capacity = " << streaming_queue_capacity << std::endl;

cv::TickMeter tm;
std::size_t frames = 0u;

cv::GMat in;
cv::GMat out      = cv::gapi::resize(in, cv::Size(), 0.5, 0.5);
cv::GMat out2      = custom::Resize::on(out, 0.2, 0.2);
cv::GMat out3      = custom::Resize2::on(out, 0.1, 0.1);
cv::GComputation graph(cv::GIn(in), cv::GOut(out2, out3));
auto kernels = cv::gapi::kernels<custom::OCVResize, custom::OCVResize2>();

{        
    auto args = cv::compile_args(kernels);
    if (streaming_queue_capacity != 0)
        args += cv::compile_args(cv::gapi::streaming::queue_capacity{ streaming_queue_capacity });

    auto pipeline = graph.compileStreaming(std::move(args));
    pipeline.setSource(cv::gin(// cv::gapi::wip::make_src<cv::gapi::wip::GCaptureSource>(input_file_name),
                               cv::gapi::wip::make_src<cv::gapi::wip::GCaptureSource>(input_file_name)
                               ));
    pipeline.start();

    cv::Mat output_frame;
    cv::Mat output_frame2;
    tm.start();
    while(pipeline.pull(cv::gout(output_frame, output_frame2))) {
        frames++;
        if(1) {
            std::cout << "pipeline process ..." << frames << std::endl;            
            std::cout << output_frame.cols << std::endl;
            std::cout << output_frame2.cols << std::endl;
        }
        // usleep(10000);
        if(frames > 5) break;
    }
    pipeline.stop();
    tm.stop();
} 

std::cout << "Processed " << frames << " frames" << " (" << frames / tm.getTimeSec() << " FPS)" << std::endl; 
return 0;

}

3

/opencv/build$ ./bin/example_test --input=./bin/cars.mp4 --queue_capacity=4cd …
streaming_queue_capacity = 4
islandActorThread 140195047405312
emitterActorThread – 140195526022912
collectorThread – 140195039012608
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …1
128
64
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …2
128
64
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …3
128
64
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …4
128
64
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …5
128
64
resize2 kernel called 140195047405312
resize kernel called 140195047405312
resize2 kernel called 140195047405312
resize kernel called 140195047405312
resize2 kernel called 140195047405312
resize kernel called 140195047405312
pipeline process …6
128
64

4

The customResize operator is called and works. But operators are executed in the same thread (islandActorThread ).

5

Hey @chris.liu ,

G-API’s execution model is defined not only by operations but also by backends managing those operations. G-API tries to “fuse” operations together into the same executable unit (“island”) if they are connected directly and are managed by the same backend - so that a backend could optimize-out this execution on its level.

In our example, you combine two Resize functions which belong to the same “OCV” backend. G-API fuses those to the same “island” (subgraph), see case #2 in the picture below - so in fact both resizes run in the same thread when executed in the streaming mode.

image
image890Ă—669 81.8 KB

The full version of the slice is @ the OpenCV Wiki: https://github.com/opencv/opencv/wiki/files/2020-09-04-GAPI_Overview.pdf

1 Like