Deep learning libraries are crucial for any successful advanced OpenCV Python application. Some are listed below (popular ones):
- PyTorch
- Keras
- TensorFlow
- Caffe
What are your opinions on each framework and the compatibility with OpenCV (what it feels like when combining these two similar things together). If there are more frameworks not listed above, please list and describe its advantages and disadvantages of any perspective. I know PyTorch and TensorFlow are talked a lot online and I’m hoping to start off with the right one to build a good application. Oh yes, I don’t anything about machine learning and just started comprehending about tensors by watching a few YouTube videos. It’s big talk for someone who doesn’t know but starting on the right path is also important it with so many opinions online. I’ve still yet to be an intermediate OpenCV programmer.
1 Like
Its been at least a year since I have looked at any of this but at that time
- PyTorch was the easiest to work with and even better if you use fastai on top of it.
- Keras older than pytorch but good for learning.
- Tensorflow was supposedly better for production but had a steeper learning curve than pytorch.
- Caffe really really old and to be avoided.
Since OpenCv can use onnx models I would say it shouldn’t matter which framework you use to train with, you just need to make sure the layers etc. have been implemented in OpenCV. At the state everything was a couple of years ago, if I was going to train deep learning models in python I would use fastai, save the models in onnx and import to opencv, but things may have changed significantly since then.
2 Likes
Caffe was a pioneer in deep learning, and it’s closed now. Don’t develop on it, but you can find some old models in this format you can use with opencv dnn.
Keras project was absorbed by TensorFlow 2 as a high level API.
Torch is closed. PyTorch, once the Python API for Torch, continues on its own. don’t know its status and future.
Tensorflow 2 seems to be the most advanced framework, you can find the some of the most recent models only in this format. If you need to start with deep learning, i believe today this is the framework for you. About it, pros and cons:
- It incorpores Keras as an entry level, a high level API, so it is easy to start with
- After Keras you can dig down through “pure” TensorFlow 2 API
- Web support with TensorFlow.JS, running TensorFlow 2 models
- It is incompatible with TensorFlow 1 (you must migrate models from 1 to 2)
- OpenCV dnn doesn’t support model files in tensorflow 2 format (you must save them as onnx to port to opencv; if you get an already trained model it can be hard, even impossible, because of new layers on TensorFlow 2)
- TensorFlow 2 API is evolving fast and there are many incompatibilities between different 2.x versions, and sometimes they are so silly keeping you wondering “Why?!”
Adding TensorFlow 2 compatibility to OpenCV dnn is hard, and it seems no one is working on it right now.
So, if you are going to build and train networks in TensorFlow 2, you can save them as onnx in order to make it work with opencv dnn. It won’t be easy if you use some advanced new layers from TensorFlow 2.
So, my preferences in 2021 are:
- To start with dnn: Keras on TensorFlow 2
- For advanced projects: TensorFlow 2
- For use in OpenCV and other frameworks: TensorFlow 2, saved as onnx
- When not possible to port TensorFlow 2 to onnx: use TensorFlow Lite as a runtime framework
I’m not really a fan of TensorFlow 2. If you are a PyTorch fan, please be nice, I didn’t mean to offend you 
1 Like
jax is the popular new thing. IDK if it’s niche. it definitely seems to compete against pytorch.
tensorflow has been dying but it’s definitely dead now.
If I understand well it is jax with flax
Can you give a paper with a model trained with flax?
there’s a difference between random papers by random researchers that happen to use tensorflow/pytorch for some mundane purpose, and high-impact papers on DL/AI from institutions that are at the forefront of DL/AI research
the number of papers cannot be used to judge jax or flax on anything but how well known they are.