By taking a closer look at the Picamera API, I finally found a way to reduce the take_picture part!!
I fact, there’s a method called “capture_continuous” that produces continuous frames.
(as you said, I guess it continuously feeds a frame buffer)
So instead of calling “cam.capture”, I just have something like:
for frame in camera.capture_continuous(rawCapture, format=“bgr”, use_video_port=True):
…
The overall time of my infinite loop has now dropped from 0.30 to 0.17 
If I could now reduce the “detectMarkers” part it would be perfect (I could may be check the DetectorParameters attributes of the detectMarkers method for that, but it does not seem to be that easy)
Anyway perfs are much better now.