Python cuda.BackgroundSubtractorMOG2 getBackgroundImage exception

[WisdomPill]

Hello,

I compiled OpenCV with CUDA because I wanted to try to offload to gpu some tasks on a jetson nano. I tried the most expensive part of my project of course, that involved background subtractor. I think I figured out how to use it from python but I cannot understand if I'm using it wrong or there's a bug. I cannot get the background image back from the background subtractor

System information (version)

• OpenCV => 4.1.1
• Operating System / Platform => Ubuntu 18.04
• Compiler => GCC

Detailed description

Traceback (most recent call last):
  File "bgsegm.py", line 20, in <module>
    background = bg_subtractor.getBackgroundImage(stream)
cv2.error: OpenCV(4.1.1) /home/anas/opencv/modules/core/src/matrix_wrap.cpp:359: error: (-213:The function/feature is not implemented) getGpuMat is available only for cuda::GpuMat and cuda::HostMem in function 'getGpuMat'

Steps to reproduce

import cv2

image = cv2.imread('assets/S11.bmp')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

gpu_gray = cv2.cuda_GpuMat()
gpu_gray.upload(gray)

bg_subtractor = cv2.cuda.createBackgroundSubtractorMOG2()

stream = cv2.cuda_Stream()

foreground_gpu = bg_subtractor.apply(gpu_gray, -1, stream)

foreground = foreground_gpu.download()


background = bg_subtractor.getBackgroundImage(stream)

[cudawarped]

This bug is a bug. It is a result of not passing the background images as an input argument. You should be able to use the below until it is fixed

background =  cv2.cuda_GpuMat(gpu_gray.size(),gpu_gray.type())
bg_subtractor.getBackgroundImage(stream, background)

[WisdomPill]

Thank you very much!

Performance seem to benefit a lot, roughly between 50 and 90% faster than the cpu one in the opencv library.

Do you know when the fix will be released?

Is it going to be in the next release?

[cudawarped]

I don't know when a fix will be released but if you build against #2396 then it should be fixed. That said for perfomance you want to pre-allocate the GpuMat arrays and pass them as input arguments. So in addition to
bg_subtractor.getBackgroundImage(stream, background)
you should also do

foreground_gpu =  cv2.cuda_GpuMat(gpu_gray.size(),gpu_gray.type())
bg_subtractor.apply(gpu_gray, -1, stream, foreground_gpu )

See accelerating opencv with cuda stream in python for an indication of the speed improvement this can result in.

Just out of interest are you using CUDA streams, or just passing them because they are a required input? If not I would suggest using
stream = cv.cuda.Stream_Null()
to avoid problems with synchronization.

[WisdomPill]

Thank you very much.

So I guess in the next release it should all be fixed.

I just started approaching gpu data processing in general, so I don't know yet if streams would benefit my code or not, I just used them because they where required.

If they are not always required I would always put Stream_Null as a default argument.

Would you say that giving the variable for the output image is always more efficient or only in the case of GpuMat?

[cudawarped]

Would you say that giving the variable for the output image is always more efficient or only in the case of GpuMat?

It avoids the output image being allocated internally in C++. Therefore it will depend on what you are doing. Most of the time you will find yourself calling the same OpenCV function several times and/or the same array will be passed to several routines. In this case it would make sense to pre-allocate the array once (even in numpy) instead of letting OpenCV allocate a new array every time a function is called.

[WisdomPill]

Sorry I forgot to reply, thanks for the suggestions by the way.

I will close the issue since you pointed out the workaround and made a pr, which is #2396 to fix it.

[Teagueporter]

import cv2 as cv2
from cv2 import cuda_Stream

cap = cv2.VideoCapture('two_minutes.mp4')
gpu_image = cv2.cuda_GpuMat()

backSub = cv2.cuda.createBackgroundSubtractorMOG2()
stream = cv2.cuda_Stream()

while True:
ret, frame = cap.read()

if frame is None:
    break

gpu_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gpu_image.upload(frame)

background = cv2.cuda_GpuMat(gpu_image.size(), gpu_image.type())

fgMask = backSub.getBackgroundImage(background, stream)

cv2.imshow("Cuda test 2 ", fgMask)

I'm running into a problem while using BS. I have a working model when using the CPU but this is the first time I am using cuda and a GPU. I tried following along what was said earlier but I'm missing something.

Traceback (most recent call last):
File "/home/teague/cudaTracker.py", line 23, in
fgMask = backSub.getBackgroundImage(background, stream)
TypeError: Expected Ptrcv::cuda::GpuMat for argument 'backgroundImage'

[cudawarped]

@Teagueporter

fgMask = backSub.getBackgroundImage(background, stream)

It looks like you are passing the arguments in the wrong order, it should be

fgMask = backSub.getBackgroundImage(stream, background)

I understand the ordering is probably not what you expect but thats because background is an optional input argument so it has to come after stream, which for some reason in this case is mandatory.

If you have any issues with the OpenCV python bindings I would first check the python tests to see if there is an example of how to call the function you want, e.g.

opencv_contrib/modules/cudabgsegm/misc/python/test/test_cudabgsegm.py

Line 34 in d502f4c

mog2.getBackgroundImage(cv.cuda.Stream_Null(),cuMatBg)

In python, before using a function I would also suggest checking the help to see the order of the arguments it expects, e.g.

help(cv2.cuda.BackgroundSubtractorMOG2.getBackgroundImage)
Help on method_descriptor:

getBackgroundImage(...)
    getBackgroundImage(stream[, backgroundImage]) -> backgroundImage
    .

If you are still having issues, then there could be problem with the python bindings, in which case I would rase and issue.