There is flag CAP_PROP_CONVERT_RGB which is supposed to turn off BGR conversion, but I think it is not implemented for FFmpeg backend. I’ve found this old PR - Support CV_CAP_MODE_GRAY in FFMPEG backend by cristiklein · Pull Request #9123 · opencv/opencv · GitHub - theoretically you should be able to port it to your OpenCV.
You can try to use GStreamer backend as it allows writing custom pipeline with custom output format (should be supported opencv/cap_gstreamer.cpp at 4b6047e746f06ce3e595c886cf6c0266498c6a67 · opencv/opencv · GitHub).