OpenCV cuda bindings for python seem receive wrong param types for cudawarping/src/warp.cpp

[MiaoDX]

System information (version)

• OpenCV => 4.1

• Operating System / Platform => Linux Ubuntu 16.04 64 Bit (Nvidia Xavier)

• CUDA => 10.x

• OpenCV => OpenCV cuda bindings for python seem receive wrong param types for cudawarping/src/warp.cpp.

Detailed description

The parameter M for warpAffine, buildWarpAffineMaps, warpPerspective, buildWarpPerspectiveMaps is all defined as cv::Mat?
However, in python binding, they are expected as cv.cuda_GpuMat. (ohterwise TypeError: Expected Ptr<cv::UMat> for argument '%s' will be raised).
However, pass cuda_GpuMat will trigger error at Mat M = _M.getMat(); with xxx/opencv/modules/core/src/matrix_wrap.cpp:118: error: (-213:The function/feature is not implemented) You should explicitly call download method for cuda::GpuMat object in function 'getMat_'

Steps to reproduce

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
#
# Demo usage of cuda.warpAffine from OpenCV 4.x

import cv2 as cv
import numpy as np

mask = np.random.randint(low=100, high=255, size=(1000, 2000)).astype(np.uint8)
mapping = np.array([[1.0, 0.0, 400.0], [0.0, 1.0, 300.0]])

image_container = cv.cuda_GpuMat()
mapping_container = cv.cuda_GpuMat()
dsize = (100, 200)
stream = cv.cuda_Stream()

image_container.upload(mask)
mapping_container.upload(mapping)

crop_mask = cv.cuda.warpAffine(src=image_container,
                              M=mapping_container,
                              dsize=dsize,
                              flags=cv.WARP_INVERSE_MAP,
                              stream=stream).download()

cv.imshow('CropMask', crop_mask)
cv.waitKey()

Possible solutions

I am not so familiar with InputArray as once-for-all param type, so, maybe this can be some easy fix for you guys. Currently, I replace Mat M = _M.getMat(); to Mat M; _M.download(M); to make python happy (and will break cpp codes..)
Another gossip, the cuda version seems no good than CPU version -.-

[cudawarped]

Hi,
Try adding the changes in #2396 to see if this resolves your problem.
For usage see this notebook where I get around 2 orders of magnitude speed up over the CPU version (Mobile RTX 2080 vs 17-8700) on a 4K video.

[perrywang]

same problem when using python binding cv2.cuda.warpPerspective

[cudawarped]

same problem when using python binding cv2.cuda.warpPerspective

is that with the changes in #2396 ?

[perrywang]

same problem when using python binding cv2.cuda.warpPerspective

is that with the changes in #2396 ?

Yes! Expect merge. I will port fix at local if it will take a little bit long wait time.

[perrywang]

@cudawarped I ported fix at local and test still error happend. Below is my simple test function
TypeError: Expected Ptrcv::UMat for argument '%s'

import cv2
import numpy as np

img = cv2.imread("1.jpg")
img_shape = (img.shape[1], img.shape[0])

img_cuda = cv2.cuda_GpuMat()
img_cuda.upload(img)

matrix = np.array([3, 3])
matrix_cuda = cv2.cuda_GpuMat()
matrix_cuda.upload(matrix)

cv2.cuda.warpPerspective(img_cuda, matrix_cuda, img_shape)

Traceback (most recent call last):
  File "cuda_demo.py", line 15, in <module>
    cv2.cuda.warpPerspective(img_cuda, matrix_cuda, img_shape)
TypeError: Expected Ptr<cv::UMat> for argument '%s'

help(cv2.cuda.warpPerspective) output

Help on built-in function warpPerspective:

warpPerspective(...)
    warpPerspective(src, M, dsize[, dst[, flags[, borderMode[, borderValue[, stream]]]]]) -> dst
    .   @brief Applies a perspective transformation to an image.
    .
    .   @param src Source image. CV_8U , CV_16U , CV_32S , or CV_32F depth and 1, 3, or 4 channels are
    .   supported.
    .   @param dst Destination image with the same type as src . The size is dsize .
    .   @param M *3x3* transformation matrix.
    .   @param dsize Size of the destination image.
    .   @param flags Combination of interpolation methods (see resize ) and the optional flag
    .   WARP_INVERSE_MAP specifying that M is the inverse transformation ( dst =\> src ). Only
    .   INTER_NEAREST , INTER_LINEAR , and INTER_CUBIC interpolation methods are supported.
    .   @param borderMode
    .   @param borderValue
    .   @param stream Stream for the asynchronous version.
    .
    .   @sa warpPerspective

[cudawarped]

@perrywang

I should have mentioned earlier that fix will not work if M is a GpuMat since in C++ the transform matrix can only be a Mat. This should probably be in the documentation.

Currently it does not make sense for M to be a GpuMat, since internally it is Mat and using a GpuMat will add the overhead of downloading and blocking on the host, if as suggested above you called
_M.download(M);
That is unless you have changed the internals to use a GpuMat without the need to transfer to the host?

With the modification in #2396 the tests in test_cudawarping.py pass locally for me.

[perrywang]

@cudawarped Got it! Many thanks!!! We are misunderstanding this method usage.

[marianbrunet]

@perrywang were you able to solve the error? I am getting the same:
TypeError: Expected Ptrcv::UMat for argument '%s'
Thanks!

[mkarzhaubayeva]

@nievesbrunet were you able to resolve this? TypeError: Expected Ptrcv::UMat for argument '%s'

[cudawarped]

@nievesbrunet were you able to resolve this? TypeError: Expected Ptrcv::UMat for argument '%s'

@mkarzhaubayeva Are you using the latest version of OpenCV? If so can you check your arguments are correct by comparing with test_cudawarping.py?

[marianbrunet]

@mkarzhaubayeva unfortunately I didn't get it to work

[cudawarped]

@mkarzhaubayeva unfortunately I didn't get it to work

@nievesbrunet Can you remember which version of OpenCV you were using and if you were using the correct arguments? For instance passing M as a GpuMat will result in the error you mentioned?

[cudawarped]

@alalek as test_cudawarping.py is passing without issue and if @nievesbrunet and/or @mkarzhaubayeva don't have a reproducable example of how this is failing on the master branch should we close this issue?