I want to compile and run a graph using the gapi module, with mimimal preprocessing, a simple net inference, without postprocessing, using tradition image I/O (not streaming). Unfortunately, there are no examples for that available. Can someone please share a working example?
this g-api tutorial covers dnn based face-detection, as well as gender/age classification
Thanks, but these examples are for video streaming. I want a very simple example, just doing one inference and compiling a graph, that can later be used to infere any cv::Mat.
indeed, it’s quite heavy on the streaming ;(
do you have OpenVino ? (it seems, it can only load ie models, bummer)
Yes, i compiled opencv (4.5.2) with inference engine (2020.04) and ngraph. I can infere the used net (an xml and a bin from OpenVINO) without gapi. However, when I try to put the inference into a graph, I get “Kernel org.opencv.dnn.infer was not found”. OS is Ubuntu 16.04.
imho, the example can be reduced for single images:
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/gapi.hpp"
#include "opencv2/gapi/core.hpp"
#include "opencv2/gapi/imgproc.hpp"
#include "opencv2/gapi/infer.hpp"
#include "opencv2/gapi/infer/ie.hpp"
#include "opencv2/gapi/cpu/gcpukernel.hpp"
#include "opencv2/gapi/streaming/cap.hpp"
const std::string about =
"This is an OpenCV-based version of Security Barrier Camera example";
const std::string keys =
"{ h help | | print this help message }"
"{ input | | Path to an input img file }"
"{ fdm | | IE face detection model IR }"
"{ fdw | | IE face detection model weights }"
"{ fdd | | IE face detection device }";
namespace custom {
//! [G_API_NET]
// Face detector: takes one Mat, returns another Mat
G_API_NET(Faces, <cv::GMat(cv::GMat)>, "face-detector");
// SSD Post-processing function - this is not a network but a kernel.
// The kernel body is declared separately, this is just an interface.
// This operation takes two Mats (detections and the source image),
// and returns a vector of ROI (filtered by a default threshold).
// Threshold (or a class to select) may become a parameter, but since
// this kernel is custom, it doesn't make a lot of sense.
G_API_OP(PostProc, <cv::GArray<cv::Rect>(cv::GMat, cv::GMat)>, "custom.fd_postproc") {
static cv::GArrayDesc outMeta(const cv::GMatDesc &, const cv::GMatDesc &) {
// This function is required for G-API engine to figure out
// what the output format is, given the input parameters.
// Since the output is an array (with a specific type),
// there's nothing to describe.
return cv::empty_array_desc();
}
};
// OpenCV-based implementation of the above kernel.
GAPI_OCV_KERNEL(OCVPostProc, PostProc) {
static void run(const cv::Mat &in_ssd_result,
const cv::Mat &in_frame,
std::vector<cv::Rect> &out_faces) {
const int MAX_PROPOSALS = 200;
const int OBJECT_SIZE = 7;
const cv::Size upscale = in_frame.size();
const cv::Rect surface({0,0}, upscale);
out_faces.clear();
const float *data = in_ssd_result.ptr<float>();
for (int i = 0; i < MAX_PROPOSALS; i++) {
const float image_id = data[i * OBJECT_SIZE + 0]; // batch id
const float confidence = data[i * OBJECT_SIZE + 2];
const float rc_left = data[i * OBJECT_SIZE + 3];
const float rc_top = data[i * OBJECT_SIZE + 4];
const float rc_right = data[i * OBJECT_SIZE + 5];
const float rc_bottom = data[i * OBJECT_SIZE + 6];
if (image_id < 0.f) { // indicates end of detections
break;
}
if (confidence < 0.5f) { // a hard-coded snapshot
continue;
}
// Convert floating-point coordinates to the absolute image
// frame coordinates; clip by the source image boundaries.
cv::Rect rc;
rc.x = static_cast<int>(rc_left * upscale.width);
rc.y = static_cast<int>(rc_top * upscale.height);
rc.width = static_cast<int>(rc_right * upscale.width) - rc.x;
rc.height = static_cast<int>(rc_bottom * upscale.height) - rc.y;
out_faces.push_back(rc & surface);
}
}
};
//! [Postproc]
} // namespace custom
int main(int argc, char *argv[])
{
cv::CommandLineParser cmd(argc, argv, keys);
cmd.about(about);
if (cmd.has("help")) {
cmd.printMessage();
return 0;
}
const std::string input = cmd.get<std::string>("input");
// Express our processing pipeline. Lambda-based constructor
// is used to keep all temporary objects in a dedicated scope.
//! [GComputation]
cv::GComputation pp([]() {
// Declare an empty GMat - the beginning of the pipeline.
cv::GMat in;
// Run face detection on the input frame. Result is a single GMat,
// internally representing an 1x1x200x7 SSD output.
// This is a single-patch version of infer:
// - Inference is running on the whole input image;
// - Image is converted and resized to the network's expected format
// automatically.
cv::GMat detections = cv::gapi::infer<custom::Faces>(in);
// Parse SSD output to a list of ROI (rectangles) using
// a custom kernel. Note: parsing SSD may become a "standard" kernel.
cv::GArray<cv::Rect> faces = custom::PostProc::on(detections, in);
// Now specify the computation's boundaries - our pipeline consumes
// one images and produces one output.
return cv::GComputation(cv::GIn(in),
cv::GOut(faces));
});
//! [GComputation]
// Note: it might be very useful to have dimensions loaded at this point!
// After our computation is defined, specify how it should be executed.
// Execution is defined by inference backends and kernel backends we use to
// compile the pipeline (it is a different step).
// Declare IE parameters for FaceDetection network. Note here custom::Face
// is the type name we specified in GAPI_NETWORK() previously.
// cv::gapi::ie::Params<> is a generic configuration description which is
// specialized to every particular network we use.
//
// OpenCV DNN backend will have its own parmater structure with settings
// relevant to OpenCV DNN module. Same applies to other possible inference
// backends...
//! [Param_Cfg]
auto det_net = cv::gapi::ie::Params<custom::Faces> {
cmd.get<std::string>("fdm"), // read cmd args: path to topology IR
cmd.get<std::string>("fdw"), // read cmd args: path to weights
cmd.get<std::string>("fdd"), // read cmd args: device specifier
};
// Form a kernel package (with a single OpenCV-based implementation of our
// post-processing) and a network package (holding our three networks).
auto kernels = cv::gapi::kernels<custom::OCVPostProc>();
auto networks = cv::gapi::networks(det_net);
{ // inference !
// Declare data objects we will be receiving from the pipeline.
cv::Mat in_frame = cv::imread("input"); // the input ! // The captured frame itself
std::vector<cv::Rect> faces; // Array of detected faces
pp.apply(cv::gin(in_frame),
cv::gout(faces),
cv::compile_args(kernels, networks));
// do something with the face rects
}
return 0;
}
however, without openvino, we get: G-API has been compiled without OpenVINO IE support
2 Likes
This code works! Thank you for figuring this out.
1 Like
Next step would be figuring out, how to serialize the whole graph and deploy it somewhere, with no prior knowledge of the details. With a simple graph, it would be the following.
// Make simple graph
cv::GMat in;
cv:GMat out = cv::gapi::resize(in, cv::Size(), 0.01, 0.01);
auto p = cv::gapi::serialize(cv::GComputation(in, out));
std::ofstream fout(“gcomp.bin”, std::ios::out | std::ios::binary);
fout.write((const char*)&p[0], p.size()); fout.close();
// Load graph
cv::Mat imgIn = cv::Mat::ones(8000, 4000, CV_8U);
cv::Mat imgOut;
std::ifstream file(“gcomp.bin”, std::ios::binary);
file.unsetf(std::ios::skipws);
std::streampos fileSize;
file.seekg(0, std::ios::end);
fileSize = file.tellg();
file.seekg(0, std::ios::beg);
std::vector<char> vec;
vec.reserve(fileSize);
vec.insert(vec.begin(), std::istream_iterator<char>(file), std::istream_iterator<char>());
auto c = cv::gapi::deserialize<cv::GComputation>(vec);
c.apply(imgIn, imgOut);
Now this cannot be applied to the upper example, as the kernels are not known by the consumer. Maybe someone is also interested in this approach. I’ll post a solution if I find one. Trying ot contact contributors and authors on this issue (G-API : how is serialize/deserialize supposed to be used properly ? · Issue #17965 · opencv/opencv · GitHub).
1 Like
Serialization has improved much since then, but may not include the inference part yet. Will follow-up on Github
2 Likes
How did you manage to compile gapi? I’m trying to compile OpenCV gapi but when I run my application I get an exception “G-API has been compiled without OpenVINO IE support”
Using cmake-gui to set params when selecting OpenVINO etc it doesn’t produce correct cmakelist cannot find something like “ocv.3rdparty.openvino”
Any help there?
i have not tried this in a while, – but what kind of model are you trying to use ?
there are indeed some, that require the openvino/IE backend
(and cannot be used without)
I’m not sure, I was given 2 .xml & .bin files to use. One is for recognizing license plates and the other faces.
Assuming they need OpenVINO, I have installed the runtime files and the dev package (although couldn’t find c++ version and only python), How do I compile OpenCV with OpenVINO support?
Because if I set through cmake-gui the engine and select support for OpenVINO I get the error I previously mentioned (ocv.3rdparty.openvino).
Do I need to download from github OpenVINO source and build it myself?
indeed, those are for IE only
wont work,imo. you need to build from src, with openvino (c++) sdk present
1 Like
Yes you’re right.
Just downloaded from github the source code of OpenVINO & tried to compile, it requires OpenCV, but in order for me to build OpenCV I require OpenVINO
Full cycle
I need G-API from OpenCV to use the OpenVINO inference engine, but I cannot build OpenVINO without OpenCV
Am I the only one who programs in plain c++ without python? I have no need to python (OpenCV or other library)
1 Like
Hey @hamlatzis.
You can download openvino archive for ubuntu20 from there: https://storage.openvinotoolkit.org/repositories/openvino/packages/2022.2/linux/l_openvino_toolkit_ubuntu20_2022.2.0.7713.af16ea1d79a_x86_64.tgz
or find for your os: Download Intel® Distribution of OpenVINO™ Toolkit
- Unpack archive and call:
source <OPENVINO_ROOT>/setupvars.sh
- Go to <OPENCV_BUILD>
- Build
cmake ../ -DWITH_INF_ENGINE=ON -DCMAKE_BUILD_TYPE=Release
make -j8 opencv_gapi
Hello @lz1004. You can find usage of cv::gapi::infer there: opencv/gapi_infer_ie_test.cpp at 4.x · opencv/opencv · GitHub
1 Like