Opencv qrcode detector crashes with out of memory exception

Python
,
1

As a part of my application, to decode qr codes, I am using cv2.QRCodeDetector() class and detectAndDecode() function.

python version is 3.7.7, opencv-version 4.2.0, platform is arm64.
Below is the output of uname -a

Linux imx8mmevk 5.4.24-2.1.0+gbabac008e5cf #1 SMP PREEMPT Mon Feb 8 13:58:43 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Below is the source code

import cv2
import time

stream = cv2.VideoCapture('v4l2src ! video/x-bayer, format=bggr, width=640, height=480 ! bayer2rgb ! videoconvert ! appsink')
qr_detector = cv2.QRCodeDetector()

def read_qr():
    try:
        while stream.isOpened():
            _, frame = stream.read()
            frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
            if frame is None:
                print('unable to read from stream')
                break

            retval, points, straight_qrcode = qr_detector.detectAndDecode(frame)
            if retval:
                print('retval', retval)
                return retval
    except Exception as ex:
        print(ex)
        return None

while True:
    read_qr()


stream.release()

Below are logs of the crash.

OpenCV(4.2.0) /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 9025000000 bytes in function ‘OutOfMemoryError’

OpenCV(4.2.0) /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 2756250000 bytes in function ‘OutOfMemoryError’

OpenCV(4.2.0) /usr/src/debug/opencv/4.2.0.imx-r0/git/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 5852250000 bytes in function ‘OutOfMemoryError’

[ 6866.719095] DEBUG END 0
[ 6884.598553] python3 invoked oom-killer: gfp_mask=0x400dc0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), order=0, oom_score_adj=0
[ 6884.611432] CPU: 1 PID: 976 Comm: python3 Not tainted 5.4.24-2.1.0+gbabac008e5cf #1
[ 6884.619099] Hardware name: iMX8MM DDR4 Honeywell TEV board (DT)
[ 6884.625026] Call trace:
[ 6884.627476] dump_backtrace+0x0/0x140
[ 6884.631137] show_stack+0x14/0x20
[ 6884.634451] dump_stack+0xb4/0xf8
[ 6884.637765] dump_header+0x44/0x1ec
[ 6884.641252] oom_kill_process+0x1d4/0x1d8
[ 6884.645260] out_of_memory+0x170/0x4e0
[ 6884.649009] __alloc_pages_slowpath+0x954/0x9f8
[ 6884.653537] __alloc_pages_nodemask+0x21c/0x280
[ 6884.658065] alloc_pages_current+0x7c/0xe8
[ 6884.662160] __pte_alloc+0x24/0x128
[ 6884.665646] __handle_mm_fault+0xe28/0x10a0
[ 6884.669826] handle_mm_fault+0xdc/0x1a8
[ 6884.673660] do_page_fault+0x130/0x460
[ 6884.677408] do_translation_fault+0x5c/0x78
[ 6884.681588] do_mem_abort+0x3c/0x98
[ 6884.685074] el0_da+0x1c/0x20
[ 6884.688085] Mem-Info:
[ 6884.690370] active_anon:319363 inactive_anon:2151 isolated_anon:0
[ 6884.690370] active_file:33 inactive_file:14 isolated_file:0
[ 6884.690370] unevictable:0 dirty:0 writeback:15 unstable:0
[ 6884.690370] slab_reclaimable:3297 slab_unreclaimable:7072
[ 6884.690370] mapped:335 shmem:2301 pagetables:797 bounce:0
[ 6884.690370] free:136985 free_pcp:104 free_cma:130546
[ 6884.723583] Node 0 active_anon:1277452kB inactive_anon:8604kB active_file:208kB inactive_file:280kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1612kB dirty:0kB writeback:60kB shmem:9204kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[ 6884.750953] Node 0 DMA32 free:547188kB min:30000kB low:37500kB high:45000kB active_anon:1277160kB inactive_anon:8604kB active_file:180kB inactive_file:488kB unevictable:0kB writepending:0kB present:2093056kB managed:2026972kB mlocked:0kB kernel_stack:1984kB pagetables:3188kB bounce:0kB free_pcp:800kB local_pcp:120kB free_cma:522184kB
[ 6884.780540] lowmem_reserve: 0 0 0
[ 6884.785106] Node 0 DMA32: 644kB (UMEC) 648kB (UMEC) 3216kB (UMEC) 4032kB (UMEC) 2564kB (UMEC) 17128kB (UMEC) 3256kB (EC) 7512kB (UMEC) 11024kB (U) 12048kB (E) 24096kB (EC) 28192kB (EC) 116384kB (C) 1532768kB (C) = 546240kB
[ 6884.806152] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[ 6884.815029] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=32768kB
[ 6884.824600] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 6884.833072] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=64kB
[ 6884.841358] 2496 total pagecache pages
[ 6884.845123] 0 pages in swap cache
[ 6884.848453] Swap cache stats: add 0, delete 0, find 0/0
[ 6884.853694] Free swap = 0kB
[ 6884.856587] Total swap = 0kB
[ 6884.859481] 523264 pages RAM
[ 6884.862364] 0 pages HighMem/MovableOnly
[ 6884.866218] 16521 pages reserved
[ 6884.869463] 163840 pages cma reserved
[ 6884.873140] 0 pages hwpoisoned
[ 6884.876207] Tasks state (memory values in pages):
[ 6884.880930] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 6884.889595] [ 371] 0 371 4455 458 61440 0 -250 systemd-journal
[ 6884.898666] [ 386] 0 386 3387 391 45056 0 -1000 systemd-udevd
[ 6884.907570] [ 668] 0 668 552 27 45056 0 0 atd
[ 6884.915601] [ 672] 0 672 785 76 45056 0 0 crond
[ 6884.923819] [ 673] 997 673 1117 112 45056 0 -900 dbus-daemon
[ 6884.932559] [ 680] 0 680 2133 162 57344 0 0 ofonod
[ 6884.940849] [ 691] 0 691 1941 775 49152 0 0 haveged
[ 6884.949226] [ 692] 0 692 539 36 40960 0 0 syslogd
[ 6884.957608] [ 697] 993 697 926 57 49152 0 0 rpcbind
[ 6884.966004] [ 703] 0 703 1925 126 49152 0 0 systemd-logind
[ 6884.974991] [ 710] 0 710 2163 1707 53248 0 0 klogd
[ 6884.983195] [ 741] 998 741 1178 62 49152 0 0 avahi-daemon
[ 6884.992008] [ 742] 998 742 1178 62 49152 0 0 avahi-daemon
[ 6885.000819] [ 743] 992 743 818 147 40960 0 0 rpc.statd
[ 6885.009375] [ 748] 990 748 18299 124 53248 0 0 ntpd
[ 6885.017493] [ 754] 0 754 1252 26 45056 0 0 agetty
[ 6885.025784] [ 755] 0 755 1140 100 45056 0 0 login
[ 6885.033999] [ 760] 0 760 2324 228 57344 0 0 systemd
[ 6885.042379] [ 761] 0 761 39758 525 77824 0 0 (sd-pam)
[ 6885.050842] [ 767] 0 767 1174 365 49152 0 0 sh
[ 6885.058786] [ 972] 0 972 644493 313516 2793472 0 0 python3
[ 6885.067162] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-0.slice/session-c2.scope,task=python3,pid=972,uid=0
[ 6885.083128] Out of memory: Killed process 972 (python3) total-vm:2577972kB, anon-rss:1254056kB, file-rss:8kB, shmem-rss:0kB, UID:0 pgtables:2728kB oom_score_adj:0
[ 6885.265492] oom_reaper: reaped process 972 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Killed

The problem is, this code runs only for few seconds and after that it crashed with out of memory exception.

Please suggest.

2

please replace the screenshot of your code with a TEXT version
(else ppl here cannot try it), thank you.

3

it’s also a very good idea to use something newer than v4.2.0

1 Like
4

updated with text version

2 Likes
5

will try that, thanks

6

speculating here…

perhaps processing happens slower than frames arrive, so frames queue up… normally, all such queues are bounded. perhaps some queue here is unbounded… and grows.

experiments:

  1. run your code without detectAndDecode. does it OOM?
  2. take a frame from the stream, keep it around, stop the stream
  3. run detectAndDecode repeatedly on that frame, without any VideoCapture. does it OOM?

that’d give some interesting data.

7

In all the three experiments, program is not crashing

8

great. my hypothesis now is that frames queue up, filling all available RAM.

I predict that this will also OOM:

import time
import cv2
stream = cv2.VideoCapture('v4l2src ! video/x-bayer, format=bggr, width=640, height=480 ! bayer2rgb ! videoconvert ! appsink')
assert stream.isOpened()
while True:
    rv, frame = stream.read()
    if not rv: break
    time.sleep(1) # a second

if my prediction is correct, you’ll have to either reduce the frame rate from your video source, or discard some frames, or decouple reading and processing by the use of threads.

9

No, crackwitz, that is not the case, this below program is not crashing. It runs without crashing for approx 3 min

image
image974×442 16.7 KB

This seems very weird, but this is my conclusion now,

Videocapture and qr code decoder , both modules individually work fine. That is what the above experiments lead to.

Where as when these two are used combinedly, something is going wrong here.

What do you think?

10

I don’t use gstreamer so I can’t reproduce this.

11

okay. Please share any other suggestions, apart from trying a newer version.
Regarding newer version, I am currently trying to include ver 4.4 on my board.

Thanks.

12

You could try an older version, in case this is an incompatibility bug just in that version

13

here’s a list of issues related to QRCodeDetector, to give you an impression of the complexity and work on the function: Issues · opencv/opencv · GitHub

there are reports of memory leaks, maybe unrelated.

14

yup, now it seems like only sensible option is to change the version of opencv.

For me, unfortunately, even, zbar library is also giving linking issue.
python 3.x - OSError while trying to import pyzbar libray on imx8m mini board - Stack Overflow

This zbar issue seems very rare, I am not able to find solutions for zbar issue also.

I have been trying to fix either of these(opencv/zbar) from past few days, but no luck yet. Will try updating to opencv 4.4.0 and see if it helps.

Thanks