Using different CUDA devices for different dnn::Net

C++
,
1

Hi.
I’m working on a project that loads several NN as dnn::Net objects were speed it’s an important factor, so I’m trying to figure out if I can spread the work to different CUDA devices.

I searched the internet and found that you can select what CUDA device you want to use with cv::cuda::setDevice but, I was wondering if this would work trying to load two different dnn::Net to different CUDA devices, would I have to call setDevice every time before calling forward in the respective dnn::Net, or once loaded with a specific device the dnn::Net will use it’s assigned device automatically. Or maybe this is not even posible with OpenCV, I can’t seem to find a answer for this.

Right now I’m unable to test this because I only have one GPU, but as mention before speed is a priority for this project so I’m trying to figure out if I could spread the work on different devices before buying another GPU.

Anyway, Thank you.

1 Like
2

That’s an interesting question, I have never tried it in the same thread but I can’t think of a reason why it wouldn’t work. It should definitely works in separate threads, at least it works 100% for standard CUDA functions.

It should be easy for you to try?

3

Thank you for your answer. I also think it would be fairly easy to test but right now I only have one graphics card and with current prices I don’t think I can get one just to test this theory.

It’s good to know that it’s probably posible when I’m able to get one.