Skip to content

Add cudastereo semi global matching (SGM) implementation #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 27, 2020
Merged

Add cudastereo semi global matching (SGM) implementation #1

merged 2 commits into from
Nov 27, 2020

Conversation

sotsuka-fixstars
Copy link

@sotsuka-fixstars sotsuka-fixstars commented Jan 7, 2019 •
edited
Loading

Porting of https://github.com/fixstars/libSGM

TODO

  • Add SGM function
  • [ ] Add samples
    • [ ] C++
    • [ ] Python lite weight sample here
  • Add tests
    • performance test
    • accuracy test
      (regression test needs extra files in opencv_extra/testdata/cv/stereomatching/algorithms
  • Add documentations
    • Doxygen
    • [ ] tutorial
  • Check coding style and License
  • Reveiw
  • Squash commits and add original authors

Re-read contribution guides and precedents

@sotsuka-fixstars sotsuka-fixstars self-assigned this Jan 7, 2019
sotsuka-fixstars
sotsuka-fixstars commented Jan 7, 2019
View reviewed changes
sotsuka-fixstars
sotsuka-fixstars commented Jan 8, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

Then, we can run program.
However, it outs empty results...

I am goint to debug.

@sotsuka-fixstars
Copy link
Author

Then, test of winner takes all will be failed.
I am going to debug winner takes all.

@sotsuka-fixstars
Copy link
Author

All our test will be passed, but
my porting show following result...

mono
colored

I am going to debug further.

@sotsuka-fixstars
Copy link
Author

I forgot DISP_SCALE at creating sample.

Now, my porting results following image.

fixed

sotsuka-fixstars
sotsuka-fixstars commented Apr 4, 2019
View reviewed changes
sotsuka-fixstars
sotsuka-fixstars commented Apr 4, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

I have ported fixstars/libSGM@b851212 .
Then, we have to fix uniqueness check like fixstars/libSGM@5b895d5 .

sotsuka-fixstars
sotsuka-fixstars commented Apr 10, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

Now, SGM, simple regression test and performance test have been implemented.
I would like to be going to write doc and sample.

@sotsuka-fixstars
Copy link
Author

The reason why we are using Center-Symmetric Census Transform is written by following paper.
https://www.mi.fu-berlin.de/inf/groups/ag-ki/publications/Semi-Global_Matching/caip2013rsp_fu.pdf

We have to add @cite in document.

@sotsuka-fixstars
Copy link
Author

I confirmed that python binding works fine by following gist.
https://gist.github.com/sotsuka-fixstars/6d2051f0a10c7a375ce969c1d5776279

sotsuka-fixstars
sotsuka-fixstars commented Apr 15, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

I am trying to use cudev module and latest version, but I get stuck because of build error.
I will try several way.

@sotsuka-fixstars sotsuka-fixstars force-pushed the develop/add_cudastereo_semi_global_matching_implementation branch from 839abde to 2d48c14 Compare April 16, 2019 03:39
@sotsuka-fixstars sotsuka-fixstars changed the base branch from add_cudastereo_semi_global_matching_implementation to master April 16, 2019 03:48
@sotsuka-fixstars sotsuka-fixstars changed the base branch from master to add_cudastereo_semi_global_matching_implementation April 16, 2019 03:49
@sotsuka-fixstars
Copy link
Author

Now, this branch rebased root master,
build goes well.

@sotsuka-fixstars
Copy link
Author

I am going to check coding style and change LICENSE.

sotsuka-fixstars
sotsuka-fixstars commented Apr 18, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

Finally, I am going to format code by refering
https://github.com/opencv/opencv/wiki/Coding_Style_Guide#code-layout

@sotsuka-fixstars
Copy link
Author

I checked that building, test, perf_test and python script work fine from scratch.

@atakagi-fixstars Could you review these?

NOTE: modules/cudastereo/samples/sample.cpp exists for check behavior.
I think that it should be going to be dropped.

@sotsuka-fixstars sotsuka-fixstars removed the request for review from atakagi-fixstars June 14, 2019 06:55
@sotsuka-fixstars
Copy link
Author

sotsuka-fixstars commented Sep 2, 2019 •
edited
Loading

For the time being, I added cuda/stereosgm_* files.
However, I don't feel that it is an appropriate way.
I would like to put all functions in cuda/stereosgbm.cu.

@sotsuka-fixstars
Copy link
Author

sotsuka-fixstars commented Sep 2, 2019 •
edited
Loading

I would like to do followings.

  • [ ] rebase to 3.4 branch modules around cuda were moved to opencv_contrib from 4.x. If we want to send PR to 3.4, we have to fork opencv/opencv repository.
  • wrap __shfl_* funcs
  • add regression test data

@sotsuka-fixstars sotsuka-fixstars force-pushed the add_cudastereo_semi_global_matching_implementation branch from fd00489 to bb5dadd Compare September 3, 2019 01:35
@sotsuka-fixstars sotsuka-fixstars force-pushed the develop/add_cudastereo_semi_global_matching_implementation branch from 1c9a2c9 to fff3b25 Compare September 3, 2019 02:03
@sotsuka-fixstars sotsuka-fixstars force-pushed the add_cudastereo_semi_global_matching_implementation branch from bb5dadd to eda4575 Compare September 3, 2019 02:03
@sotsuka-fixstars sotsuka-fixstars force-pushed the develop/add_cudastereo_semi_global_matching_implementation branch 2 times, most recently from dca3027 to 8a751c1 Compare September 3, 2019 02:12
@sotsuka-fixstars
Copy link
Author

I tested cv::cuda::StereoSGM in following CMake settings.

  • CUDA 10.1, CUDA_ARCH_BIN=6.1
  • CUDA 8.0, CUDA_ARCH_BIN=6.1
  • CUDA 8.0, CUDA_ARCH_PTX=2.0

opencv_test_cudastereo project and python sample is used for test.

@sotsuka-fixstars
Copy link
Author

827d910 Added regression test using params and previous result.
However, this commit has too duplicate codes to
https://github.com/opencv/opencv/blob/fc35c77f00219cf62eed63867240b10c6a7b1a26/modules/calib3d/test/test_stereomatching.cpp
Furthermore, we have to send PR to opencv/opencv_extra for adding these files.

@sotsuka-fixstars
Copy link
Author

sotsuka-fixstars commented Sep 4, 2019 •
edited
Loading

To check regression test, I compared

  1. Ground Truth
  2. cv::StereoSGBM (numDisp=32, winSize=3, mode=MODE_SGBM)
  3. cv::cuda::StereoSGM (numDisp=64, mode=MODE_HH)
data (1) (2) (3)
barn2 ground_truth middleburu_2001_barn2 cv_stereosgbm middleburu_2001_barn2 cv_cuda_stereosgm middleburu_2001_barn2
cones ground_truth middleburu_2003_cones cv_stereosgbm middleburu_2003_cones cv_cuda_stereosgm middleburu_2003_cones
tsukuba ground_truth tsukuba cv_stereosgbm tsukuba cv_cuda_stereosgm tsukuba

In allmost quality metrics, cv::cuda::StereoSGM wins cv::StereoSGBM.
However, cv::cuda::StereoSGM is not good at depth discontinuity regions. It means that cv::cuda::StereoSGM cannot preserve edges better than cv::StereoSGBM.
Moreover, it is clear that tsukuba task is difficult for cv::cuda::StereoSGM from resulting image.

@sotsuka-fixstars sotsuka-fixstars changed the title [WIP] Add cudastereo semi global matching (SGM) implementation Add cudastereo semi global matching (SGM) implementation Sep 9, 2019
atakagi-fixstars
atakagi-fixstars reviewed Sep 9, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

I have fixed code reviewed.
I also re-checked code and test going passed.

atakagi-fixstars
atakagi-fixstars reviewed Sep 9, 2019
View reviewed changes
Copy link

@atakagi-fixstars atakagi-fixstars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some reviews

atakagi-fixstars
atakagi-fixstars reviewed Sep 11, 2019
View reviewed changes
Copy link

@atakagi-fixstars atakagi-fixstars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a review.

sotsuka-fixstars
sotsuka-fixstars commented Sep 12, 2019
View reviewed changes
@sotsuka-fixstars
Copy link
Author

sotsuka-fixstars commented Sep 13, 2019 •
edited
Loading

I handled warnings, and checked tests passing, python sample running and code difference.
Sry for late.

@sotsuka-fixstars sotsuka-fixstars force-pushed the develop/add_cudastereo_semi_global_matching_implementation branch from 38d6ace to cb78723 Compare October 2, 2020 10:58
@sotsuka-fixstars sotsuka-fixstars force-pushed the develop/add_cudastereo_semi_global_matching_implementation branch from cb78723 to fcf4fa2 Compare November 20, 2020 02:41
@sotsuka-fixstars sotsuka-fixstars changed the base branch from add_cudastereo_semi_global_matching_implementation to master November 20, 2020 02:48
@sotsuka-fixstars sotsuka-fixstars changed the base branch from master to add_cudastereo_semi_global_matching_implementation November 20, 2020 02:48
atakagi-fixstars
atakagi-fixstars reviewed Nov 27, 2020
View reviewed changes
@atakagi-fixstars atakagi-fixstars merged commit 453e256 into add_cudastereo_semi_global_matching_implementation Nov 27, 2020
sotsuka-fixstars pushed a commit that referenced this pull request Nov 27, 2020
@atakagi-fixstars @sotsuka-fixstars
Merge pull request #1 from fixstars/develop/add_cudastereo_semi_globa…
043d670 
…l_matching_implementation

Add cudastereo semi global matching (SGM) implementation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@atakagi-fixstars atakagi-fixstars atakagi-fixstars left review comments

Assignees

@sotsuka-fixstars sotsuka-fixstars

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sotsuka-fixstars @atakagi-fixstars