How to Filesink and Appsink simultaneously in OpenCV Gstreamer. (Gstreamer pipeline included)

,
1

Hi,
I’m developing C++ based multi v4l2 camera application on Jetson AGX Xavier.

Current Status

[1 V4L2, 2 V4L2 ... N V4L2 ] >-----[App]-----> [Output]

It requires N x cameras’ original videos and one more processed output video.

  1. original video appsink pipeline

    gstream_elements = "v4l2src device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)BGRx, width=(int)3840, height=(int)2160 ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink" and it is reading the camera input like this below.
    cv::VideoCapture(gstream_elements, cv::CAP_GSTREAMER)

  2. original video encoding pipeline & Result Output video encoding pipeline

    gstream_elements = "appsrc ! video/x-raw, format=(string)BGR ! queue ! videoconvert ! video/x-raw, format(string)I420 ! queue ! nvvidconv ! video/x-raw(memory:NVMM), width=3840, height=2160 ! queue ! omxh264enc qp-range=15,30:5,20:-1,-1 ! queue ! mpegtsmux ! hlssink max-files=0 playlist-length=0 target-duration=4 playlist-location=0playlist.m3u8 location=0segment%05d.ts " and it writes the frame like this below.
    cv::VideoWriter(gstream_elements, cv::CAP_GSTREAMER, 0, m_fps, cv::Size(3840, 2160), true)

Issue

  1. Current separated pipeline show HIGH CPU USAGE. Therefore, I want to integrate appsink and filesink in one pipeline. but it seems it doesn’t work. when I search on web, it maybe because opencv VideoCapture cannot do both job…
    Is there any other way?

    gstream_elements = "v4l2src device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160 ! nvvidconv ! tee name=t ! queue ! video/xraw(memory:NVMM), format=(string)I420, width=(int)3840, height=(int)2160 ! omxh264enc qp-range=20,30:20,30:-1,-1 ! queue ! mpegtsmux ! hlssink max-files=0 playlist-length=0 target-duration=4 playlist-location=0playlist.m3u8 location=0segment%05d.ts t. ! queue ! video/x-raw(memory:NVMM), format=(string)BGRx, width=(int)3840, height=(int)2160 ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink" cv::VideoCapture(gstream_elements, cv::CAP_GSTREAMER)

  2. Some says ipcpipeline can be used here. but it failes to get ipcpipelinesrc.

    video_capture_gstream_elements = "v4l2src device=/dev/video0 ! video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160 ! nvvidconv ! tee name=t ! queue ! ipcpipelinesink t. ! queue ! video/x-raw(memory:NVMM), format=(string)BGRx, width=(int)3840, height=(int)2160 ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink" video_writer_gstream_elements = "ipcpipelinesrc ! video/xraw(memory:NVMM), format=(string)I420, width=(int)3840, height=(int)2160 ! omxh264enc qp-range=20,30:20,30:-1,-1 ! queue ! mpegtsmux ! hlssink max-files=0 playlist-length=0 target-duration=4 playlist-location=0playlist.m3u8 location=0segment%05d.ts "

is there somebody who succeeded in appsink and filesink at the same time?

Thank you.

2

‘tee’ works fine.
but when there is heavy computing load, some pipeline is stopped.