diff --git a/modules/alphamat/README.md b/modules/alphamat/README.md new file mode 100644 index 00000000000..9d91b48e2f4 --- /dev/null +++ b/modules/alphamat/README.md @@ -0,0 +1,60 @@ +# Designing Effective Inter-Pixel Information Flow for Natural Image Matting: + +Alphamatting is the problem of extracting the foreground from an image. Given the input of image and its corresponding trimap, we try to extract the foreground from the background. Following is an example - + +Input Image | Input trimap | Ouput Alpha matte +:-------------------------:|:-------------------------:|:-------------------------: +alt text | alt text | alt text + +This project is implementation of Information-Flow Matting [Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys] [1]. It required implementation of some parts of other papers [2,3]. + +This is a pixel-affinity based alpha matting algorithm which solves a linear system of equations using preconditioned conjugate gradient method. Affinity-based methods operate by propagating opacity information from known opacity regions(K) into unknown opacity regions(U) using a variety of affinity definitions mentioned as - +* Color mixture information flow - Opacity transitions in a matte occur as a result of the original colors in the image getting mixed with each other due to transparency or intricate parts of an object. They make use of this fact by representing each pixel in U as a mixture of similarly-colored pixels and the difference is the energy term ECM, which is to be reduced. This is coded in **cm.hpp** +* K-to-U information flow - Connections from every pixel in U to both F(foreground pixels) and B(background pixels) are made to facilitate direct information flow from known-opacity regions to even the most remote opacity-transition regions in the image. This is coded in **KtoU.hpp** +* Intra U information flow - They distribute the information inside U effectively by encouraging pixels with similar colors inside U to have similar opacity. This is coded in **intraU.hpp** +* Local information flow - Spatial connectivity is one of the main cues for information flow which is achieved by connecting each pixel in U to its immediate neighbors to ensure spatially smooth mattes. This is coded in **local_info.hpp** + +Using these information flow, energy/error(E) is obtained as a weighted local composite of ECM, EKU(K-to-U information flow), EUU(Intra U information flow), EL(Local information flow). +E represents the deviation of unknown pixels opacity or colour from what we predict it to be using other pixels. So, the algorithm aims at minimizing this error. This is coded in **alphac.cpp** + +Pre-processing and post-processing is implemented in **trimming.hpp** + +To run the code - +1. **g++ -std=c++11 alphac.cpp \`pkg-config --cflags --libs opencv\`** +1. **./a.out \ \** + +Sample image and trimap are in opencv_contrib/modules/alphamat/src/img and opencv_contrib/modules/alphamat/src/trimap + +## Results + +Results for input_lowres are available here - +https://docs.google.com/document/d/1BJG4633_U5K-Z0QLp3RTi43q25NI0hrTw-Q4w_85NrA/edit?usp=sharing + +Input Image | Ouput Alpha matte +:-------------------------:|:-------------------------: +alt text | alt text +alt text | alt text +alt text | alt text +alt text | alt text +alt text | alt text +alt text | alt text +alt text | alt text +alt text | alt text + +Average time taken to compute the different flows is 40s, but solving of linear equations using preconditioned conjugate gradient method takes another 2-3 min, which can be lessened by allowing lesser iterations. + +## TO DO + +* Results need to be improved by extensively comparing each flow's matrix with yaksoy MATLAB implementation [4]. +* Runtime needs improvement. +* Third part library(Eigen, nanoflann) dependencies can be removed. + +## References + +[1] Yagiz Aksoy, Tunc Ozan Aydin, Marc Pollefeys, "Designing Effective Inter-Pixel Information Flow for Natural Image Matting", CVPR, 2017. [[link](http://people.inf.ethz.ch/aksoyy/ifm/)] + +[2] Roweis, Sam T., and Lawrence K. Saul. "Nonlinear dimensionality reduction by locally linear embedding." science 290.5500 (2000): 2323-2326.[[link](https://science.sciencemag.org/content/290/5500/2323)] + +[3] Ehsan Shahrian, Deepu Rajan, Brian Price, Scott Cohen, "Improving Image Matting using Comprehensive Sampling Sets", CVPR 2013 [[paper](http://www.cv-foundation.org/openaccess/content_cvpr_2013/papers/Shahrian_Improving_Image_Matting_2013_CVPR_paper.pdf)] + +[4] Affinity Based Matting Toolbox by Yagiz Aksoy[[link](https://github.com/yaksoy/AffinityBasedMattingToolbox)] diff --git a/modules/alphamat/Result/result_doll.png b/modules/alphamat/Result/result_doll.png new file mode 100644 index 00000000000..562bac50dca Binary files /dev/null and b/modules/alphamat/Result/result_doll.png differ diff --git a/modules/alphamat/Result/result_donkey.png b/modules/alphamat/Result/result_donkey.png new file mode 100644 index 00000000000..78b99c60dd7 Binary files /dev/null and b/modules/alphamat/Result/result_donkey.png differ diff --git a/modules/alphamat/Result/result_elephant.png b/modules/alphamat/Result/result_elephant.png new file mode 100644 index 00000000000..eca9043e0d7 Binary files /dev/null and b/modules/alphamat/Result/result_elephant.png differ diff --git a/modules/alphamat/Result/result_net.png b/modules/alphamat/Result/result_net.png new file mode 100644 index 00000000000..9b0b2b7723a Binary files /dev/null and b/modules/alphamat/Result/result_net.png differ diff --git a/modules/alphamat/Result/result_pineapple.png b/modules/alphamat/Result/result_pineapple.png new file mode 100644 index 00000000000..4732e451b9a Binary files /dev/null and b/modules/alphamat/Result/result_pineapple.png differ diff --git a/modules/alphamat/Result/result_plant.png b/modules/alphamat/Result/result_plant.png new file mode 100644 index 00000000000..54932dacacf Binary files /dev/null and b/modules/alphamat/Result/result_plant.png differ diff --git a/modules/alphamat/Result/result_plasticbag.png b/modules/alphamat/Result/result_plasticbag.png new file mode 100644 index 00000000000..fd286394c84 Binary files /dev/null and b/modules/alphamat/Result/result_plasticbag.png differ diff --git a/modules/alphamat/Result/result_troll.png b/modules/alphamat/Result/result_troll.png new file mode 100644 index 00000000000..d140dd94c4b Binary files /dev/null and b/modules/alphamat/Result/result_troll.png differ diff --git a/modules/alphamat/img/doll.png b/modules/alphamat/img/doll.png new file mode 100644 index 00000000000..806752bc84d Binary files /dev/null and b/modules/alphamat/img/doll.png differ diff --git a/modules/alphamat/img/donkey.png b/modules/alphamat/img/donkey.png new file mode 100644 index 00000000000..7b97e58ff89 Binary files /dev/null and b/modules/alphamat/img/donkey.png differ diff --git a/modules/alphamat/img/elephant.png b/modules/alphamat/img/elephant.png new file mode 100644 index 00000000000..32a09c145d5 Binary files /dev/null and b/modules/alphamat/img/elephant.png differ diff --git a/modules/alphamat/img/net.png b/modules/alphamat/img/net.png new file mode 100644 index 00000000000..0a42be2809d Binary files /dev/null and b/modules/alphamat/img/net.png differ diff --git a/modules/alphamat/img/pineapple.png b/modules/alphamat/img/pineapple.png new file mode 100644 index 00000000000..d491ec546e0 Binary files /dev/null and b/modules/alphamat/img/pineapple.png differ diff --git a/modules/alphamat/img/plant.png b/modules/alphamat/img/plant.png new file mode 100644 index 00000000000..ffb8c4722d4 Binary files /dev/null and b/modules/alphamat/img/plant.png differ diff --git a/modules/alphamat/img/plasticbag.png b/modules/alphamat/img/plasticbag.png new file mode 100644 index 00000000000..07a3a704920 Binary files /dev/null and b/modules/alphamat/img/plasticbag.png differ diff --git a/modules/alphamat/img/troll.png b/modules/alphamat/img/troll.png new file mode 100644 index 00000000000..9bdc64e661d Binary files /dev/null and b/modules/alphamat/img/troll.png differ diff --git a/modules/alphamat/perf/perf_infoflow.cpp b/modules/alphamat/perf/perf_infoflow.cpp new file mode 100644 index 00000000000..65efcfe148f --- /dev/null +++ b/modules/alphamat/perf/perf_infoflow.cpp @@ -0,0 +1,39 @@ +using namespace std; +using namespace cv; +using namespace perf; + +#include "perf_precomp.hpp" + +namespace opencv_test +{ + +typedef std::tr1::tuple Size_MatType_OutMatDepth_t; +typedef perf::TestBaseWithParam Size_MatType_OutMatDepth; + +/* 2. Declare the testsuite */ +PERF_TEST_P( Size_MatType_OutMatDepth, integral1, + testing::Combine( + testing::Values( TYPICAL_MAT_SIZES ), + testing::Values( CV_8UC1, CV_8UC4 ), + testing::Values( CV_32S, CV_32F, CV_64F ) ) ) +{ + string folder = "cv/alphamat/"; + string image_path = folder + "img/elephant.png"; + string trimap_path = folder + "trimap/elephant.png"; + string reference_path = folder + "reference/elephant.png"; + + Mat image = imread(getDataPath(image_path), IMREAD_COLOR); + Mat trimap = imread(getDataPath(trimap_path), IMREAD_COLOR); + Mat reference = imread(getDataPath(reference_path), IMREAD_GRAYSCALE); + + Size sz = get<0>(GetParam()); + int inpaintingMethod = get<1>(GetParam()); + + Mat result; + declare.in(image, trimap).out(result).time(120); + + TEST_CYCLE() infoFlow(image, trimap, result, false, true); + + SANITY_CHECK_NOTHING(); +} +} // namespace diff --git a/modules/alphamat/perf/perf_main.cpp b/modules/alphamat/perf/perf_main.cpp new file mode 100644 index 00000000000..607449352ac --- /dev/null +++ b/modules/alphamat/perf/perf_main.cpp @@ -0,0 +1,7 @@ +#include "perf_precomp.hpp" + +#if defined(HAVE_HPX) + #include +#endif + +CV_PERF_TEST_MAIN(stitching) \ No newline at end of file diff --git a/modules/alphamat/perf/perf_precomp.hpp b/modules/alphamat/perf/perf_precomp.hpp new file mode 100644 index 00000000000..bd62b2d0149 --- /dev/null +++ b/modules/alphamat/perf/perf_precomp.hpp @@ -0,0 +1,6 @@ +#ifndef __OPENCV_PERF_PRECOMP_HPP__ +#define __OPENCV_PERF_PRECOMP_HPP__ + +#include "opencv2/ts.hpp" + +#endif diff --git a/modules/alphamat/src/KDTreeVectorOfVectorsAdaptor.h b/modules/alphamat/src/KDTreeVectorOfVectorsAdaptor.h new file mode 100644 index 00000000000..83870f09983 --- /dev/null +++ b/modules/alphamat/src/KDTreeVectorOfVectorsAdaptor.h @@ -0,0 +1,116 @@ +/*********************************************************************** + * Software License Agreement (BSD License) + * + * Copyright 2011-16 Jose Luis Blanco (joseluisblancoc@gmail.com). + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + *************************************************************************/ + +#pragma once + +#include "nanoflann.hpp" + +#include + +// ===== This example shows how to use nanoflann with these types of containers: ======= +//typedef std::vector > my_vector_of_vectors_t; +//typedef std::vector my_vector_of_vectors_t; // This requires #include +// ===================================================================================== + + +/** A simple vector-of-vectors adaptor for nanoflann, without duplicating the storage. + * The i'th vector represents a point in the state space. + * + * \tparam DIM If set to >0, it specifies a compile-time fixed dimensionality for the points in the data set, allowing more compiler optimizations. + * \tparam num_t The type of the point coordinates (typically, double or float). + * \tparam Distance The distance metric to use: nanoflann::metric_L1, nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. + * \tparam IndexType The type for indices in the KD-tree index (typically, size_t of int) + */ +template +struct KDTreeVectorOfVectorsAdaptor +{ + typedef KDTreeVectorOfVectorsAdaptor self_t; + typedef typename Distance::template traits::distance_t metric_t; + typedef nanoflann::KDTreeSingleIndexAdaptor< metric_t, self_t, DIM, IndexType> index_t; + + index_t* index; //! The kd-tree index for the user to call its methods as usual with any other FLANN index. + + /// Constructor: takes a const ref to the vector of vectors object with the data points + KDTreeVectorOfVectorsAdaptor(const size_t /* dimensionality */, const VectorOfVectorsType &mat, const int leaf_max_size = 10) : m_data(mat) + { + assert(mat.size() != 0 && mat[0].size() != 0); + const size_t dims = mat[0].size(); + if (DIM>0 && static_cast(dims) != DIM) + throw std::runtime_error("Data set dimensionality does not match the 'DIM' template argument"); + index = new index_t( static_cast(dims), *this /* adaptor */, nanoflann::KDTreeSingleIndexAdaptorParams(leaf_max_size ) ); + index->buildIndex(); + } + + ~KDTreeVectorOfVectorsAdaptor() { + delete index; + } + + const VectorOfVectorsType &m_data; + + /** Query for the \a num_closest closest points to a given point (entered as query_point[0:dim-1]). + * Note that this is a short-cut method for index->findNeighbors(). + * The user can also call index->... methods as desired. + * \note nChecks_IGNORED is ignored but kept for compatibility with the original FLANN interface. + */ + inline void query(const num_t *query_point, const size_t num_closest, IndexType *out_indices, num_t *out_distances_sq, const int nChecks_IGNORED = 10) const + { + nanoflann::KNNResultSet resultSet(num_closest); + resultSet.init(out_indices, out_distances_sq); + index->findNeighbors(resultSet, query_point, nanoflann::SearchParams()); + } + + /** @name Interface expected by KDTreeSingleIndexAdaptor + * @{ */ + + const self_t & derived() const { + return *this; + } + self_t & derived() { + return *this; + } + + // Must return the number of data points + inline size_t kdtree_get_point_count() const { + return m_data.size(); + } + + // Returns the dim'th component of the idx'th point in the class: + inline num_t kdtree_get_pt(const size_t idx, const size_t dim) const { + return m_data[idx][dim]; + } + + // Optional bounding-box computation: return false to default to a standard bbox computation loop. + // Return true if the BBOX was already computed by the class and returned in "bb" so it can be avoided to redo it again. + // Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3 for point clouds) + template + bool kdtree_get_bbox(BBOX & /*bb*/) const { + return false; + } + + /** @} */ +}; // end of KDTreeVectorOfVectorsAdaptor diff --git a/modules/alphamat/src/KtoU.hpp b/modules/alphamat/src/KtoU.hpp new file mode 100644 index 00000000000..5d1ed1e32c1 --- /dev/null +++ b/modules/alphamat/src/KtoU.hpp @@ -0,0 +1,274 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +// #ifndef KtoU +// #define KtoU + +#include +#include +#include + +#include +#include "nanoflann.hpp" +#include "KDTreeVectorOfVectorsAdaptor.h" +#include "Eigen/Sparse" +using namespace Eigen; +using namespace nanoflann; +using namespace std; +using namespace cv; +// const int dim = 5; + +typedef vector> my_vector_of_vectors_t; +typedef vector>> my_vector_of_set_t; +map, int> ind; //original index mapping set - (unk, fg, bg) + +void generateFVectorKtoU(my_vector_of_vectors_t& fv_unk, my_vector_of_vectors_t& fv_fg, my_vector_of_vectors_t& fv_bg, + Mat &img, Mat &tmap) +{ + // CV_Assert(img.depth() == CV_8U); + int nRows = img.rows; + int nCols = img.cols; + + int fg = 0, bg = 0, unk = 0, c1 = 0, c2 = 0, c3 = 0; + int i, j, k; + for (i = 0; i < nRows; ++i) + for (j = 0; j < nCols; ++j){ + double pix = tmap.at(i, j); + if (pix == 128) + unk++; + else if (pix > 200) + fg++; + else + bg++; + } + + fv_fg.resize(fg); + fv_bg.resize(bg); + fv_unk.resize(unk); + + for (i = 0; i < nRows; ++i) + for (j = 0; j < nCols; ++j){ + double pix = tmap.at(i, j); + if (pix == 128){ + fv_unk[c1].resize(dim); + fv_unk[c1][0] = img.at(i, j)[0]/255.0; + fv_unk[c1][1] = img.at(i, j)[1]/255.0; + fv_unk[c1][2] = img.at(i, j)[2]/255.0; + fv_unk[c1][3] = double(i)*10/nRows; + fv_unk[c1][4] = double(j)*10/nCols; + ind[{c1, 0}] = i*nCols+j; + c1++; + }else if (pix > 200){ + fv_fg[c2].resize(dim); + fv_fg[c2][0] = img.at(i, j)[0]/255.0; + fv_fg[c2][1] = img.at(i, j)[1]/255.0; + fv_fg[c2][2] = img.at(i, j)[2]/255.0; + fv_fg[c2][3] = double(i)*10/nRows; + fv_fg[c2][4] = double(j)*10/nCols; + ind[{c2, 1}] = i*nCols+j; + c2++; + }else{ + fv_bg[c3].resize(dim); + fv_bg[c3][0] = img.at(i, j)[0]/255.0; + fv_bg[c3][1] = img.at(i, j)[1]/255.0; + fv_bg[c3][2] = img.at(i, j)[2]/255.0; + fv_bg[c3][3] = double(i)*10/nRows; + fv_bg[c3][4] = double(j)*10/nCols; + ind[{c3,2}] = i*nCols+j; + c3++; + } + } + + // cout << "feature vectors done "< my_kd_tree_t; + my_kd_tree_t mat_index_fg(dim /*dim*/, fv_fg, 10 /* max leaf */ ); + mat_index_fg.index->buildIndex(); + + my_kd_tree_t mat_index_bg(dim /*dim*/, fv_bg, 10 /* max leaf */ ); + mat_index_bg.index->buildIndex(); + + // do a knn search with cm = 20 + const size_t num_results = 7; + + int N = fv_unk.size(); + + vector ret_indexes(num_results); + vector out_dists_sqr(num_results); + nanoflann::KNNResultSet resultSet(num_results); + + indm.resize(N); + for (int i = 0; i < fv_unk.size(); i++){ + indm[i].resize(2*num_results); + + resultSet.init(&ret_indexes[0], &out_dists_sqr[0] ); + mat_index_fg.index->findNeighbors(resultSet, &fv_unk[i][0], nanoflann::SearchParams(10)); + for (int j = 0; j < num_results; j++){ + // cout << "$$$$$$$ret_index["<(p, j) = fv_fg[index_nbr][p] - fv_unk[i][p]; + } + + for (j = k/2; j < k; j++){ + index_nbr = indm[i][j]; + for (p = 0; p < dim-2; p++) + Z.at(p, j) = fv_bg[index_nbr][p] - fv_unk[i][p]; + } + // cout<<"ours\n"; + // cout<(p, p) += eps; + // cout<<"determinant: "<(j, 0); + // cout<<"SUM:"<(j, 0) /= sum; + // nbr_ind = indm[i][j]; + // triplets.push_back(T(ind[{i, 0}], ind[{nbr_ind, 1}], weights.at(j, 0))); + } + + for (int j = k/2; j < k; j++){ + weights.at(j, 0) /= sum; + // nbr_ind = indm[i][j]; + // triplets.push_back(T(ind[{i, 0}], ind[{nbr_ind, 2}], weights.at(j, 0))); + } + + + // calculating confidence values + double fweight = 0, bweight = 0, nu = 0; + double fcol[3], bcol[3]; + for (j = 0; j < 3; j++){ + fcol[j] = 0; + bcol[j] = 0; + } + for (j = 0; j < k/2; j++){ + fweight += weights.at(j, 0); + index_nbr = indm[i][j]; + for (p = 0; p < dim-2; p++) + fcol[p] += weights.at(j, 0) * fv_fg[index_nbr][p]; + } + + for (j = k/2; j < k; j++){ + bweight += weights.at(j, 0); + index_nbr = indm[i][j]; + for (p = 0; p < dim-2; p++) + bcol[p] += weights.at(j, 0) * fv_bg[index_nbr][p]; + } + + double norm; + for (j = 0; j < 3; j++){ + norm = fcol[j]/fweight - bcol[j]/bweight; + nu += norm * norm; + } + + // cout<(ind[{i, 0}], 0) = fweight; + } + + // Wku.setFromTriplets(triplets.begin(), triplets.end()); + H.setFromTriplets(tripletsH.begin(), tripletsH.end()); + return H; +} + + +SparseMatrix KtoU(Mat& image, Mat& tmap, Mat& wf){ + my_vector_of_vectors_t fv_fg, fv_bg, fv_unk, indm, Euu; + + int i, j; + kdtree_KtoU(image, tmap, indm, fv_unk, fv_fg, fv_bg); + // cout<<"KD Tree done"< H = lle_KtoU(indm, fv_unk, fv_fg, fv_bg, eps, wf); + cout << "KToU Done" << endl; + return H; +} + + +/* +int main() +{ + Mat image,tmap; + string img_path = "../../data/input_lowres/plasticbag.png"; + image = imread(img_path, CV_LOAD_IMAGE_COLOR); // Read the file + + string tmap_path = "../../data/trimap_lowres/Trimap1/plasticbag.png"; + tmap = imread(tmap_path, CV_LOAD_IMAGE_GRAYSCALE); + SparseMatrix ret = KtoU(image, tmap); + +} +*/ + +// #endif diff --git a/modules/alphamat/src/alphac.cpp b/modules/alphamat/src/alphac.cpp new file mode 100644 index 00000000000..6a9eaf2e277 --- /dev/null +++ b/modules/alphamat/src/alphac.cpp @@ -0,0 +1,206 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +const int dim = 5; // dimension of feature vectors +#include "precomp.hpp" + +using namespace std; +using namespace cv; + +void show(Mat& image){ + namedWindow( "Display window", WINDOW_AUTOSIZE ); // Create a window for display. + imshow( "Display window", image ); // Show our image inside it. + waitKey(0); // Wait for a keystroke in the window +} + +int check_image(Mat& image){ + if ( !image.data ) // Check for invalid input + { + cout << "Could not open or find the image" << endl; + return -1; + } + return 0; +} + +void type2str(int type) { + string r; + + uchar depth = type & CV_MAT_DEPTH_MASK; + uchar chans = 1 + (type >> CV_CN_SHIFT); + + switch ( depth ) { + case CV_8U: r = "8U"; break; + case CV_8S: r = "8S"; break; + case CV_16U: r = "16U"; break; + case CV_16S: r = "16S"; break; + case CV_32S: r = "32S"; break; + case CV_32F: r = "32F"; break; + case CV_64F: r = "64F"; break; + default: r = "User"; break; + } + + r += "C"; + r += (chans+'0'); + cout << r << endl; +} + +void solve(SparseMatrix Wcm,SparseMatrix Wuu,SparseMatrix Wl,SparseMatrix Dcm, + SparseMatrix Duu,SparseMatrix Dl,SparseMatrix H,SparseMatrix T, + Mat &ak, Mat &wf, bool useKU, Mat &alpha){ + + float sku = 0.05, suu = 0.01, sl = 1, lamd = 100; + // float sku = 0, suu = 0, sl = 0, lamd = 100; + + SparseMatrix Lifm = ((Dcm-Wcm).transpose())*(Dcm-Wcm) + suu*(Duu-Wuu) + sl*(Dl-Wl); + // # Lifm = suu*(Duu-Wuu) + sl*(Dl-Wl) + SparseMatrix A; + int n = ak.cols; + VectorXd b(n), wf_(n), x(n); + + + for (int i = 0; i < n; i++) + wf_(i) = wf.at(i, 0); + + + if (useKU) + { + A = Lifm + lamd*T + sku*H; + b = (lamd*T + sku*H)*(wf_); + }else{ + A = Lifm + lamd*T; + b = (lamd*T)*(wf_); + } + + ConjugateGradient, Lower|Upper> cg; + cg.setMaxIterations(500); + cg.compute(A); + x = cg.solve(b); + + std::cout << "#iterations: " << cg.iterations() << std::endl; + std::cout << "estimated error: " << cg.error() << std::endl; + + int nRows = alpha.rows; + int nCols = alpha.cols; + for (int i = 0; i < nRows; ++i) + for (int j = 0; j < nCols; ++j){ + // cout<(i, j) = x(i*nCols+j)*255; + } + // show(alpha); + cout << "Done" << endl; +} + + +void infoFlow(Mat& image, Mat& tmap, Mat& result, bool useKU, bool trim){ + clock_t begin = clock(); + int nRows = image.rows; + int nCols = image.cols; + int N = nRows*nCols; + + Mat ak, wf; + SparseMatrix T(N, N); + typedef Triplet Tr; + vector triplets; + // triplets.reserve(N*N); + + ak.create(1, nRows*nCols, CV_8U); + wf.create(nRows*nCols, 1, CV_8U); + for (int i = 0; i < nRows; ++i) + for (int j = 0; j < nCols; ++j){ + float pix = tmap.at(i, j); + if (pix != 128) // collection of known pixels samples + triplets.push_back(Tr(i*nCols+j, i*nCols+j, 1)); + else + triplets.push_back(Tr(i*nCols+j, i*nCols+j, 0)); + if (pix > 200) // foreground pixel + ak.at(0, i*nCols+j) = 1; + else + ak.at(0, i*nCols+j) = 0; + wf.at(i*nCols+j, 0) = ak.at(0, i*nCols+j); + } + + + SparseMatrix Wl(N, N), Dl(N, N); + local_info(image, tmap, Wl, Dl); + + SparseMatrix Wcm(N, N), Dcm(N, N); + cm(image, tmap, Wcm, Dcm); + + Mat new_tmap = tmap.clone(); // after pre-processing + // trimming(image, tmap, new_tmap, tmap, true); + trimming(image, tmap, new_tmap, trim); + SparseMatrix H = KtoU(image, new_tmap, wf); + + SparseMatrix Wuu(N, N), Duu(N, N); + UU(image, tmap, Wuu, Duu); + + + clock_t end = clock(); + double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC; + cout << "time for flow calc: " << elapsed_secs << endl; + + T.setFromTriplets(triplets.begin(), triplets.end()); + // Mat calc_alpha = solve(Wcm,Wuu,Wl,Dcm,Duu,Dl,H,T,ak,wf,true); + + Mat alpha; + alpha.create(nRows, nCols, CV_8UC1); + solve(Wcm, Wuu, Wl, Dcm, Duu, Dl, H, T, ak, wf, useKU, alpha); + + Mat trim_alpha = alpha.clone(); + cout << "Solved" << endl; + + int i, j; + for (i = 0; i < image.rows; i++){ + for (j = 0; j < image.cols; j++){ + float pix = new_tmap.at(i, j); + if (pix != 128){ + // cout<<"in "<(i, j))<(i, j) = pix; + } + if (float(trim_alpha.at(i, j)) > 250) + trim_alpha.at(i, j) = 255; + if (float(trim_alpha.at(i, j)) < 5) + trim_alpha.at(i, j) = 0; + } + } + + // trim_alpha[trim_alpha > 230] = 255; + // trim_alpha[trim_alpha < 20] = 0; + + + // cout<<"Trimmed"< +#include +#include + +#include +#include "nanoflann.hpp" +#include "KDTreeVectorOfVectorsAdaptor.h" +#include "Eigen/Sparse" +using namespace Eigen; +using namespace nanoflann; +using namespace std; +using namespace cv; + + +typedef vector> my_vector_of_vectors_t; +// typedef vector>> my_vector_of_set_t; + + +void generateFVectorCM(my_vector_of_vectors_t &samples, Mat &img) +{ + // CV_Assert(img.depth() == CV_8U); + + int channels = img.channels(); + int nRows = img.rows; + int nCols = img.cols; + + samples.resize(nRows*nCols); + + int i, j, k; + for (i = 0; i < nRows; ++i) + for (j = 0; j < nCols; ++j){ + samples[i*nCols+j].resize(dim); + samples[i*nCols+j][0] = img.at(i, j)[0]/255.0; + samples[i*nCols+j][1] = img.at(i, j)[1]/255.0; + samples[i*nCols+j][2] = img.at(i, j)[2]/255.0; + samples[i*nCols+j][3] = double(i)/nRows; + samples[i*nCols+j][4] = double(j)/nCols; + } + + // cout << "feature vectors done"<& unk) +{ + // Generate feature vectors for intra U: + generateFVectorCM(samples, img); + + // Query point: same as samples from which KD tree is generated + + // construct a kd-tree index: + // Dimensionality set at run-time (default: L2) + // ------------------------------------------------------------ + typedef KDTreeVectorOfVectorsAdaptor< my_vector_of_vectors_t, double > my_kd_tree_t; + my_kd_tree_t mat_index(dim /*dim*/, samples, 10 /* max leaf */ ); + mat_index.index->buildIndex(); + + // do a knn search with cm = 20 + const size_t num_results = 20+1; + + int N = unk.size(); + + vector ret_indexes(num_results); + vector out_dists_sqr(num_results); + nanoflann::KNNResultSet resultSet(num_results); + + indm.resize(N); + int i = 0; + for (unordered_set::iterator it = unk.begin(); it != unk.end(); it++){ + resultSet.init(&ret_indexes[0], &out_dists_sqr[0] ); + mat_index.index->findNeighbors(resultSet, &samples[*it][0], nanoflann::SearchParams(10)); + + // cout << "knnSearch(nn="<::iterator it = unk.begin(); it != unk.end(); it++){ + // filling values in Z + i = *it; + int index_nbr; + for (int j = 0; j < k; j++){ + index_nbr = indm[ind][j]; + for (int p = 0; p < dim-2; p++) + Z.at(p, j) = samples[index_nbr][p] - samples[i][p]; + } + + + // C1 = Z1.transpose()*Z1; + // C1.diagonal().array() += eps; + // weights1 = C1.ldlt().solve(rhs1); + // weights1 /= weights1.sum(); + // cout<(p, p) += eps; + // cout<<"determinant: "<(j, 0);; + // cout<<"SUM:"<(j, 0) /= sum; + triplets.push_back(T(i, indm[ind][j], weights.at(j, 0))); + // if(ind == 0){ + // cout<(j,0)<(j, 0))); + } + // cout<(j,0); + + // } + + ind++; + } + + Wcm.setFromTriplets(triplets.begin(), triplets.end()); + Dcm.setFromTriplets(td.begin(), td.end()); + // return Wcm; +} + +void cm(Mat& image, Mat& tmap, SparseMatrix& Wcm, SparseMatrix& Dcm){ + my_vector_of_vectors_t samples, indm, Euu; + + int i, j; + unordered_set unk; + for (i = 0; i < tmap.rows; i++) + for (j = 0; j < tmap.cols; j++){ + float pix = tmap.at(i, j); + if (pix == 128) + unk.insert(i*tmap.cols+j); + } + + // cout<<"UNK: "< Wcm(N,N), Dcm(N,N); + cm(image, tmap, Wcm, Dcm); +} +*/ +// #endif diff --git a/modules/alphamat/src/infoflow.cpp b/modules/alphamat/src/infoflow.cpp new file mode 100644 index 00000000000..6a9eaf2e277 --- /dev/null +++ b/modules/alphamat/src/infoflow.cpp @@ -0,0 +1,206 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +const int dim = 5; // dimension of feature vectors +#include "precomp.hpp" + +using namespace std; +using namespace cv; + +void show(Mat& image){ + namedWindow( "Display window", WINDOW_AUTOSIZE ); // Create a window for display. + imshow( "Display window", image ); // Show our image inside it. + waitKey(0); // Wait for a keystroke in the window +} + +int check_image(Mat& image){ + if ( !image.data ) // Check for invalid input + { + cout << "Could not open or find the image" << endl; + return -1; + } + return 0; +} + +void type2str(int type) { + string r; + + uchar depth = type & CV_MAT_DEPTH_MASK; + uchar chans = 1 + (type >> CV_CN_SHIFT); + + switch ( depth ) { + case CV_8U: r = "8U"; break; + case CV_8S: r = "8S"; break; + case CV_16U: r = "16U"; break; + case CV_16S: r = "16S"; break; + case CV_32S: r = "32S"; break; + case CV_32F: r = "32F"; break; + case CV_64F: r = "64F"; break; + default: r = "User"; break; + } + + r += "C"; + r += (chans+'0'); + cout << r << endl; +} + +void solve(SparseMatrix Wcm,SparseMatrix Wuu,SparseMatrix Wl,SparseMatrix Dcm, + SparseMatrix Duu,SparseMatrix Dl,SparseMatrix H,SparseMatrix T, + Mat &ak, Mat &wf, bool useKU, Mat &alpha){ + + float sku = 0.05, suu = 0.01, sl = 1, lamd = 100; + // float sku = 0, suu = 0, sl = 0, lamd = 100; + + SparseMatrix Lifm = ((Dcm-Wcm).transpose())*(Dcm-Wcm) + suu*(Duu-Wuu) + sl*(Dl-Wl); + // # Lifm = suu*(Duu-Wuu) + sl*(Dl-Wl) + SparseMatrix A; + int n = ak.cols; + VectorXd b(n), wf_(n), x(n); + + + for (int i = 0; i < n; i++) + wf_(i) = wf.at(i, 0); + + + if (useKU) + { + A = Lifm + lamd*T + sku*H; + b = (lamd*T + sku*H)*(wf_); + }else{ + A = Lifm + lamd*T; + b = (lamd*T)*(wf_); + } + + ConjugateGradient, Lower|Upper> cg; + cg.setMaxIterations(500); + cg.compute(A); + x = cg.solve(b); + + std::cout << "#iterations: " << cg.iterations() << std::endl; + std::cout << "estimated error: " << cg.error() << std::endl; + + int nRows = alpha.rows; + int nCols = alpha.cols; + for (int i = 0; i < nRows; ++i) + for (int j = 0; j < nCols; ++j){ + // cout<(i, j) = x(i*nCols+j)*255; + } + // show(alpha); + cout << "Done" << endl; +} + + +void infoFlow(Mat& image, Mat& tmap, Mat& result, bool useKU, bool trim){ + clock_t begin = clock(); + int nRows = image.rows; + int nCols = image.cols; + int N = nRows*nCols; + + Mat ak, wf; + SparseMatrix T(N, N); + typedef Triplet Tr; + vector triplets; + // triplets.reserve(N*N); + + ak.create(1, nRows*nCols, CV_8U); + wf.create(nRows*nCols, 1, CV_8U); + for (int i = 0; i < nRows; ++i) + for (int j = 0; j < nCols; ++j){ + float pix = tmap.at(i, j); + if (pix != 128) // collection of known pixels samples + triplets.push_back(Tr(i*nCols+j, i*nCols+j, 1)); + else + triplets.push_back(Tr(i*nCols+j, i*nCols+j, 0)); + if (pix > 200) // foreground pixel + ak.at(0, i*nCols+j) = 1; + else + ak.at(0, i*nCols+j) = 0; + wf.at(i*nCols+j, 0) = ak.at(0, i*nCols+j); + } + + + SparseMatrix Wl(N, N), Dl(N, N); + local_info(image, tmap, Wl, Dl); + + SparseMatrix Wcm(N, N), Dcm(N, N); + cm(image, tmap, Wcm, Dcm); + + Mat new_tmap = tmap.clone(); // after pre-processing + // trimming(image, tmap, new_tmap, tmap, true); + trimming(image, tmap, new_tmap, trim); + SparseMatrix H = KtoU(image, new_tmap, wf); + + SparseMatrix Wuu(N, N), Duu(N, N); + UU(image, tmap, Wuu, Duu); + + + clock_t end = clock(); + double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC; + cout << "time for flow calc: " << elapsed_secs << endl; + + T.setFromTriplets(triplets.begin(), triplets.end()); + // Mat calc_alpha = solve(Wcm,Wuu,Wl,Dcm,Duu,Dl,H,T,ak,wf,true); + + Mat alpha; + alpha.create(nRows, nCols, CV_8UC1); + solve(Wcm, Wuu, Wl, Dcm, Duu, Dl, H, T, ak, wf, useKU, alpha); + + Mat trim_alpha = alpha.clone(); + cout << "Solved" << endl; + + int i, j; + for (i = 0; i < image.rows; i++){ + for (j = 0; j < image.cols; j++){ + float pix = new_tmap.at(i, j); + if (pix != 128){ + // cout<<"in "<(i, j))<(i, j) = pix; + } + if (float(trim_alpha.at(i, j)) > 250) + trim_alpha.at(i, j) = 255; + if (float(trim_alpha.at(i, j)) < 5) + trim_alpha.at(i, j) = 0; + } + } + + // trim_alpha[trim_alpha > 230] = 255; + // trim_alpha[trim_alpha < 20] = 0; + + + // cout<<"Trimmed"< +#include +#include + +#include +#include "nanoflann.hpp" +#include "KDTreeVectorOfVectorsAdaptor.h" +#include "Eigen/Sparse" +using namespace Eigen; +using namespace nanoflann; +using namespace std; +using namespace cv; + + +typedef vector> my_vector_of_vectors_t; +typedef vector>> my_vector_of_set_t; +vector orig_ind; + +void generateFVectorIntraU(my_vector_of_vectors_t &samples, Mat &img, Mat& tmap) +{ + // CV_Assert(img.depth() == CV_8U); + int channels = img.channels(); + int nRows = img.rows; + int nCols = img.cols; + int unk_count = 0; + int i, j, k; + for (i = 0; i < nRows; ++i) + for (j = 0; j < nCols; ++j){ + float pix = tmap.at(i, j); + if (pix == 128) + unk_count++; + } + samples.resize(unk_count); + orig_ind.resize(unk_count); + + int c1 = 0; + for (i = 0; i < nRows; ++i) + for (j = 0; j < nCols; ++j){ + float pix = tmap.at(i, j); + if (pix == 128){ // collection of unknown pixels samples + samples[c1].resize(dim); + samples[c1][0] = img.at(i, j)[0]; + samples[c1][1] = img.at(i, j)[1]; + samples[c1][2] = img.at(i, j)[2]; + samples[c1][3] = (double(i)/nRows)/20; + samples[c1][4] = (double(j)/nCols)/20; + orig_ind[c1] = i*nCols+j; + c1++; + } + } + + // cout << "feature vectors done"< my_kd_tree_t; + my_kd_tree_t mat_index(dim /*dim*/, samples, 10 /* max leaf */ ); + mat_index.index->buildIndex(); + // do a knn search with ku = 5 + const size_t num_results = 5+1; + + int i, j; + int N = samples.size(); // no. of unknown samples + + // just for testing purpose ...delete this later! + int c = 0; + + vector ret_indexes(num_results); + vector out_dists_sqr(num_results); + nanoflann::KNNResultSet resultSet(num_results); + + + + indm.resize(N); + inds.resize(N); + for (i = 0; i < N; i++){ + resultSet.init(&ret_indexes[0], &out_dists_sqr[0] ); + mat_index.index->findNeighbors(resultSet, &samples[i][0], nanoflann::SearchParams(10)); + + // cout << "knnSearch(nn="<& Wuu, SparseMatrix& Duu){ + my_vector_of_vectors_t samples, indm, Euu; + my_vector_of_set_t inds; + // string img_path = "../../data/input_lowres/plasticbag.png"; + // image = imread(img_path, CV_LOAD_IMAGE_COLOR); // Read the file + + // string tmap_path = "../../data/trimap_lowres/Trimap1/plasticbag.png"; + // tmap = imread(tmap_path, CV_LOAD_IMAGE_GRAYSCALE); + + kdtree_intraU(image, tmap, indm, inds, samples); + // cout<<"KD Tree done"< Wuu = UU(image, tmap); + +} + +*/ diff --git a/modules/alphamat/src/local_info.hpp b/modules/alphamat/src/local_info.hpp new file mode 100644 index 00000000000..2cb33f9c30c --- /dev/null +++ b/modules/alphamat/src/local_info.hpp @@ -0,0 +1,158 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +// #ifndef local_info +// #define local_info + +#include +#include + +#include +#include "Eigen/Sparse" +using namespace Eigen; +using namespace std; +using namespace cv; + +// const int dim = 5; + +// void show(Mat& image){ +// namedWindow( "Display window", WINDOW_AUTOSIZE );// Create a window for display. +// imshow( "Display window", image ); // Show our image inside it. +// waitKey(0); // Wait for a keystroke in the window +// } + + +void local_info(Mat& img, Mat& tmap, SparseMatrix& Wl, SparseMatrix& Dl){ + float eps = 0.001; + int win_size = 1; + + int channels = img.channels(); + int nRows = img.rows; + int nCols = img.cols; + Mat unk_img; + unk_img.create(nRows, nCols, CV_32FC1); + // Mat unk_img1 = Mat::zeros(cv::Size(2,5), CV_8U); + // cout<(i, j); + if (pix == 128){ // collection of unknown pixels samples + unk_img.at(i, j) = 255; + } + } + + // cout< Wl(N,N), Dl(N,N); + typedef Triplet T; + vector triplets, td; + + int x[] = {-1, -1, -1, 0, 0, 0, 1, 1, 1}; + int y[] = {-1, 0, 1, -1, 0, 1, -1, 0, 1}; + + int i, j; + for (i = win_size; i < img.rows-win_size; i++){ + for (j = win_size; j < img.cols-win_size; j++){ + if ((int)dilation_dst.at(i, j) == 0) { + continue; + } + + // extract the window out of image + Mat win = img.rowRange(i-win_size, i+win_size+1); + win = win.colRange(j-win_size, j+win_size+1); + Mat win_ravel = Mat::zeros(9, 3, CV_64F); // doubt ?? + double sum1 = 0; + double sum2 = 0; + double sum3 = 0; + + int c = 0; + for (int p = 0; p < win_size*2+1; p++){ + for (int q = 0; q < win_size*2+1; q++){ + win_ravel.at(c, 0) = win.at(p, q)[0]/255.0; + win_ravel.at(c, 1) = win.at(p, q)[1]/255.0; + win_ravel.at(c, 2) = win.at(p, q)[2]/255.0; + // cout<(p,q)[0])<(p, q)[0]/255.0; + sum2 += win.at(p, q)[1]/255.0; + sum3 += win.at(p, q)[2]/255.0; + c++; + } + } + win = win_ravel; + + Mat win_mean = Mat::zeros(1, 3, CV_64F); + win_mean.at(0, 0) = sum1/num_win; + win_mean.at(0, 1) = sum2/num_win; + win_mean.at(0, 2) = sum3/num_win; + + // calculate the covariance matrix + Mat covariance = (win.t() * win / num_win) - (win_mean.t() * win_mean); + + Mat I = Mat::eye(img.channels(), img.channels(), CV_64F); + Mat inv = (covariance + eps / num_win * I).inv(); + + Mat X = win - repeat(win_mean, num_win, 1); + Mat vals = (1 + X * inv * X.t()) / num_win; + vals = vals.reshape(0, num_win_sq); + + int nbr_r, nbr_c; // nrb row and col + for (int p = 0; p < num_win; p++){ + for (int q = 0; q < num_win; q++){ + nbr_r = i+x[p]; + nbr_c = j+y[p]; + triplets.push_back(T(nbr_r, nbr_c, vals.at(p*num_win+q, 0))); + td.push_back(T(nbr_r, nbr_r, vals.at(p*num_win+q, 0))); + } + } + // cout< Wl(N,N), Dl(N,N); + local_info(image, tmap, Wl, Dl); + +} +*/ + +// #endif diff --git a/modules/alphamat/src/nanoflann.hpp b/modules/alphamat/src/nanoflann.hpp new file mode 100644 index 00000000000..0e228993ab2 --- /dev/null +++ b/modules/alphamat/src/nanoflann.hpp @@ -0,0 +1,2039 @@ +/*********************************************************************** + * Software License Agreement (BSD License) + * + * Copyright 2008-2009 Marius Muja (mariusm@cs.ubc.ca). All rights reserved. + * Copyright 2008-2009 David G. Lowe (lowe@cs.ubc.ca). All rights reserved. + * Copyright 2011-2016 Jose Luis Blanco (joseluisblancoc@gmail.com). + * All rights reserved. + * + * THE BSD LICENSE + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + *************************************************************************/ + +/** \mainpage nanoflann C++ API documentation + * nanoflann is a C++ header-only library for building KD-Trees, mostly + * optimized for 2D or 3D point clouds. + * + * nanoflann does not require compiling or installing, just an + * #include in your code. + * + * See: + * - C++ API organized by modules + * - Online README + * - Doxygen + * documentation + */ + +#ifndef NANOFLANN_HPP_ +#define NANOFLANN_HPP_ + +#include +#include +#include +#include // for abs() +#include // for fwrite() +#include // for abs() +#include +#include // std::reference_wrapper +#include +#include + +/** Library version: 0xMmP (M=Major,m=minor,P=patch) */ +#define NANOFLANN_VERSION 0x130 + +// Avoid conflicting declaration of min/max macros in windows headers +#if !defined(NOMINMAX) && \ + (defined(_WIN32) || defined(_WIN32_) || defined(WIN32) || defined(_WIN64)) +#define NOMINMAX +#ifdef max +#undef max +#undef min +#endif +#endif + +namespace nanoflann { +/** @addtogroup nanoflann_grp nanoflann C++ library for ANN + * @{ */ + +/** the PI constant (required to avoid MSVC missing symbols) */ +template T pi_const() { + return static_cast(3.14159265358979323846); +} + +/** + * Traits if object is resizable and assignable (typically has a resize | assign + * method) + */ +template struct has_resize : std::false_type {}; + +template +struct has_resize().resize(1), 0)> + : std::true_type {}; + +template struct has_assign : std::false_type {}; + +template +struct has_assign().assign(1, 0), 0)> + : std::true_type {}; + +/** + * Free function to resize a resizable object + */ +template +inline typename std::enable_if::value, void>::type +resize(Container &c, const size_t nElements) { + c.resize(nElements); +} + +/** + * Free function that has no effects on non resizable containers (e.g. + * std::array) It raises an exception if the expected size does not match + */ +template +inline typename std::enable_if::value, void>::type +resize(Container &c, const size_t nElements) { + if (nElements != c.size()) + throw std::logic_error("Try to change the size of a std::array."); +} + +/** + * Free function to assign to a container + */ +template +inline typename std::enable_if::value, void>::type +assign(Container &c, const size_t nElements, const T &value) { + c.assign(nElements, value); +} + +/** + * Free function to assign to a std::array + */ +template +inline typename std::enable_if::value, void>::type +assign(Container &c, const size_t nElements, const T &value) { + for (size_t i = 0; i < nElements; i++) + c[i] = value; +} + +/** @addtogroup result_sets_grp Result set classes + * @{ */ +template +class KNNResultSet { +public: + typedef _DistanceType DistanceType; + typedef _IndexType IndexType; + typedef _CountType CountType; + +private: + IndexType *indices; + DistanceType *dists; + CountType capacity; + CountType count; + +public: + inline KNNResultSet(CountType capacity_) + : indices(0), dists(0), capacity(capacity_), count(0) {} + + inline void init(IndexType *indices_, DistanceType *dists_) { + indices = indices_; + dists = dists_; + count = 0; + if (capacity) + dists[capacity - 1] = (std::numeric_limits::max)(); + } + + inline CountType size() const { return count; } + + inline bool full() const { return count == capacity; } + + /** + * Called during search to add an element matching the criteria. + * @return true if the search should be continued, false if the results are + * sufficient + */ + inline bool addPoint(DistanceType dist, IndexType index) { + CountType i; + for (i = count; i > 0; --i) { +#ifdef NANOFLANN_FIRST_MATCH // If defined and two points have the same + // distance, the one with the lowest-index will be + // returned first. + if ((dists[i - 1] > dist) || + ((dist == dists[i - 1]) && (indices[i - 1] > index))) { +#else + if (dists[i - 1] > dist) { +#endif + if (i < capacity) { + dists[i] = dists[i - 1]; + indices[i] = indices[i - 1]; + } + } else + break; + } + if (i < capacity) { + dists[i] = dist; + indices[i] = index; + } + if (count < capacity) + count++; + + // tell caller that the search shall continue + return true; + } + + inline DistanceType worstDist() const { return dists[capacity - 1]; } +}; + +/** operator "<" for std::sort() */ +struct IndexDist_Sorter { + /** PairType will be typically: std::pair */ + template + inline bool operator()(const PairType &p1, const PairType &p2) const { + return p1.second < p2.second; + } +}; + +/** + * A result-set class used when performing a radius based search. + */ +template +class RadiusResultSet { +public: + typedef _DistanceType DistanceType; + typedef _IndexType IndexType; + +public: + const DistanceType radius; + + std::vector> &m_indices_dists; + + inline RadiusResultSet( + DistanceType radius_, + std::vector> &indices_dists) + : radius(radius_), m_indices_dists(indices_dists) { + init(); + } + + inline void init() { clear(); } + inline void clear() { m_indices_dists.clear(); } + + inline size_t size() const { return m_indices_dists.size(); } + + inline bool full() const { return true; } + + /** + * Called during search to add an element matching the criteria. + * @return true if the search should be continued, false if the results are + * sufficient + */ + inline bool addPoint(DistanceType dist, IndexType index) { + if (dist < radius) + m_indices_dists.push_back(std::make_pair(index, dist)); + return true; + } + + inline DistanceType worstDist() const { return radius; } + + /** + * Find the worst result (furtherest neighbor) without copying or sorting + * Pre-conditions: size() > 0 + */ + std::pair worst_item() const { + if (m_indices_dists.empty()) + throw std::runtime_error("Cannot invoke RadiusResultSet::worst_item() on " + "an empty list of results."); + typedef + typename std::vector>::const_iterator + DistIt; + DistIt it = std::max_element(m_indices_dists.begin(), m_indices_dists.end(), + IndexDist_Sorter()); + return *it; + } +}; + +/** @} */ + +/** @addtogroup loadsave_grp Load/save auxiliary functions + * @{ */ +template +void save_value(FILE *stream, const T &value, size_t count = 1) { + fwrite(&value, sizeof(value), count, stream); +} + +template +void save_value(FILE *stream, const std::vector &value) { + size_t size = value.size(); + fwrite(&size, sizeof(size_t), 1, stream); + fwrite(&value[0], sizeof(T), size, stream); +} + +template +void load_value(FILE *stream, T &value, size_t count = 1) { + size_t read_cnt = fread(&value, sizeof(value), count, stream); + if (read_cnt != count) { + throw std::runtime_error("Cannot read from file"); + } +} + +template void load_value(FILE *stream, std::vector &value) { + size_t size; + size_t read_cnt = fread(&size, sizeof(size_t), 1, stream); + if (read_cnt != 1) { + throw std::runtime_error("Cannot read from file"); + } + value.resize(size); + read_cnt = fread(&value[0], sizeof(T), size, stream); + if (read_cnt != size) { + throw std::runtime_error("Cannot read from file"); + } +} +/** @} */ + +/** @addtogroup metric_grp Metric (distance) classes + * @{ */ + +struct Metric {}; + +/** Manhattan distance functor (generic version, optimized for + * high-dimensionality data sets). Corresponding distance traits: + * nanoflann::metric_L1 \tparam T Type of the elements (e.g. double, float, + * uint8_t) \tparam _DistanceType Type of distance variables (must be signed) + * (e.g. float, double, int64_t) + */ +template +struct L1_Adaptor { + typedef T ElementType; + typedef _DistanceType DistanceType; + + const DataSource &data_source; + + L1_Adaptor(const DataSource &_data_source) : data_source(_data_source) {} + + inline DistanceType evalMetric(const T *a, const size_t b_idx, size_t size, + DistanceType worst_dist = -1) const { + DistanceType result = DistanceType(); + const T *last = a + size; + const T *lastgroup = last - 3; + size_t d = 0; + + /* Process 4 items with each loop for efficiency. */ + while (a < lastgroup) { + const DistanceType diff0 = + std::abs(a[0] - data_source.kdtree_get_pt(b_idx, d++)); + const DistanceType diff1 = + std::abs(a[1] - data_source.kdtree_get_pt(b_idx, d++)); + const DistanceType diff2 = + std::abs(a[2] - data_source.kdtree_get_pt(b_idx, d++)); + const DistanceType diff3 = + std::abs(a[3] - data_source.kdtree_get_pt(b_idx, d++)); + result += diff0 + diff1 + diff2 + diff3; + a += 4; + if ((worst_dist > 0) && (result > worst_dist)) { + return result; + } + } + /* Process last 0-3 components. Not needed for standard vector lengths. */ + while (a < last) { + result += std::abs(*a++ - data_source.kdtree_get_pt(b_idx, d++)); + } + return result; + } + + template + inline DistanceType accum_dist(const U a, const V b, const size_t) const { + return std::abs(a - b); + } +}; + +/** Squared Euclidean distance functor (generic version, optimized for + * high-dimensionality data sets). Corresponding distance traits: + * nanoflann::metric_L2 \tparam T Type of the elements (e.g. double, float, + * uint8_t) \tparam _DistanceType Type of distance variables (must be signed) + * (e.g. float, double, int64_t) + */ +template +struct L2_Adaptor { + typedef T ElementType; + typedef _DistanceType DistanceType; + + const DataSource &data_source; + + L2_Adaptor(const DataSource &_data_source) : data_source(_data_source) {} + + inline DistanceType evalMetric(const T *a, const size_t b_idx, size_t size, + DistanceType worst_dist = -1) const { + DistanceType result = DistanceType(); + const T *last = a + size; + const T *lastgroup = last - 3; + size_t d = 0; + + /* Process 4 items with each loop for efficiency. */ + while (a < lastgroup) { + const DistanceType diff0 = a[0] - data_source.kdtree_get_pt(b_idx, d++); + const DistanceType diff1 = a[1] - data_source.kdtree_get_pt(b_idx, d++); + const DistanceType diff2 = a[2] - data_source.kdtree_get_pt(b_idx, d++); + const DistanceType diff3 = a[3] - data_source.kdtree_get_pt(b_idx, d++); + result += diff0 * diff0 + diff1 * diff1 + diff2 * diff2 + diff3 * diff3; + a += 4; + if ((worst_dist > 0) && (result > worst_dist)) { + return result; + } + } + /* Process last 0-3 components. Not needed for standard vector lengths. */ + while (a < last) { + const DistanceType diff0 = *a++ - data_source.kdtree_get_pt(b_idx, d++); + result += diff0 * diff0; + } + return result; + } + + template + inline DistanceType accum_dist(const U a, const V b, const size_t) const { + return (a - b) * (a - b); + } +}; + +/** Squared Euclidean (L2) distance functor (suitable for low-dimensionality + * datasets, like 2D or 3D point clouds) Corresponding distance traits: + * nanoflann::metric_L2_Simple \tparam T Type of the elements (e.g. double, + * float, uint8_t) \tparam _DistanceType Type of distance variables (must be + * signed) (e.g. float, double, int64_t) + */ +template +struct L2_Simple_Adaptor { + typedef T ElementType; + typedef _DistanceType DistanceType; + + const DataSource &data_source; + + L2_Simple_Adaptor(const DataSource &_data_source) + : data_source(_data_source) {} + + inline DistanceType evalMetric(const T *a, const size_t b_idx, + size_t size) const { + DistanceType result = DistanceType(); + for (size_t i = 0; i < size; ++i) { + const DistanceType diff = a[i] - data_source.kdtree_get_pt(b_idx, i); + result += diff * diff; + } + return result; + } + + template + inline DistanceType accum_dist(const U a, const V b, const size_t) const { + return (a - b) * (a - b); + } +}; + +/** SO2 distance functor + * Corresponding distance traits: nanoflann::metric_SO2 + * \tparam T Type of the elements (e.g. double, float) + * \tparam _DistanceType Type of distance variables (must be signed) (e.g. + * float, double) orientation is constrained to be in [-pi, pi] + */ +template +struct SO2_Adaptor { + typedef T ElementType; + typedef _DistanceType DistanceType; + + const DataSource &data_source; + + SO2_Adaptor(const DataSource &_data_source) : data_source(_data_source) {} + + inline DistanceType evalMetric(const T *a, const size_t b_idx, + size_t size) const { + return accum_dist(a[size - 1], data_source.kdtree_get_pt(b_idx, size - 1), + size - 1); + } + + /** Note: this assumes that input angles are already in the range [-pi,pi] */ + template + inline DistanceType accum_dist(const U a, const V b, const size_t) const { + DistanceType result = DistanceType(), PI = pi_const(); + result = b - a; + if (result > PI) + result -= 2 * PI; + else if (result < -PI) + result += 2 * PI; + return result; + } +}; + +/** SO3 distance functor (Uses L2_Simple) + * Corresponding distance traits: nanoflann::metric_SO3 + * \tparam T Type of the elements (e.g. double, float) + * \tparam _DistanceType Type of distance variables (must be signed) (e.g. + * float, double) + */ +template +struct SO3_Adaptor { + typedef T ElementType; + typedef _DistanceType DistanceType; + + L2_Simple_Adaptor distance_L2_Simple; + + SO3_Adaptor(const DataSource &_data_source) + : distance_L2_Simple(_data_source) {} + + inline DistanceType evalMetric(const T *a, const size_t b_idx, + size_t size) const { + return distance_L2_Simple.evalMetric(a, b_idx, size); + } + + template + inline DistanceType accum_dist(const U a, const V b, const size_t idx) const { + return distance_L2_Simple.accum_dist(a, b, idx); + } +}; + +/** Metaprogramming helper traits class for the L1 (Manhattan) metric */ +struct metric_L1 : public Metric { + template struct traits { + typedef L1_Adaptor distance_t; + }; +}; +/** Metaprogramming helper traits class for the L2 (Euclidean) metric */ +struct metric_L2 : public Metric { + template struct traits { + typedef L2_Adaptor distance_t; + }; +}; +/** Metaprogramming helper traits class for the L2_simple (Euclidean) metric */ +struct metric_L2_Simple : public Metric { + template struct traits { + typedef L2_Simple_Adaptor distance_t; + }; +}; +/** Metaprogramming helper traits class for the SO3_InnerProdQuat metric */ +struct metric_SO2 : public Metric { + template struct traits { + typedef SO2_Adaptor distance_t; + }; +}; +/** Metaprogramming helper traits class for the SO3_InnerProdQuat metric */ +struct metric_SO3 : public Metric { + template struct traits { + typedef SO3_Adaptor distance_t; + }; +}; + +/** @} */ + +/** @addtogroup param_grp Parameter structs + * @{ */ + +/** Parameters (see README.md) */ +struct KDTreeSingleIndexAdaptorParams { + KDTreeSingleIndexAdaptorParams(size_t _leaf_max_size = 10) + : leaf_max_size(_leaf_max_size) {} + + size_t leaf_max_size; +}; + +/** Search options for KDTreeSingleIndexAdaptor::findNeighbors() */ +struct SearchParams { + /** Note: The first argument (checks_IGNORED_) is ignored, but kept for + * compatibility with the FLANN interface */ + SearchParams(int checks_IGNORED_ = 32, float eps_ = 0, bool sorted_ = true) + : checks(checks_IGNORED_), eps(eps_), sorted(sorted_) {} + + int checks; //!< Ignored parameter (Kept for compatibility with the FLANN + //!< interface). + float eps; //!< search for eps-approximate neighbours (default: 0) + bool sorted; //!< only for radius search, require neighbours sorted by + //!< distance (default: true) +}; +/** @} */ + +/** @addtogroup memalloc_grp Memory allocation + * @{ */ + +/** + * Allocates (using C's malloc) a generic type T. + * + * Params: + * count = number of instances to allocate. + * Returns: pointer (of type T*) to memory buffer + */ +template inline T *allocate(size_t count = 1) { + T *mem = static_cast(::malloc(sizeof(T) * count)); + return mem; +} + +/** + * Pooled storage allocator + * + * The following routines allow for the efficient allocation of storage in + * small chunks from a specified pool. Rather than allowing each structure + * to be freed individually, an entire pool of storage is freed at once. + * This method has two advantages over just using malloc() and free(). First, + * it is far more efficient for allocating small objects, as there is + * no overhead for remembering all the information needed to free each + * object or consolidating fragmented memory. Second, the decision about + * how long to keep an object is made at the time of allocation, and there + * is no need to track down all the objects to free them. + * + */ + +const size_t WORDSIZE = 16; +const size_t BLOCKSIZE = 8192; + +class PooledAllocator { + /* We maintain memory alignment to word boundaries by requiring that all + allocations be in multiples of the machine wordsize. */ + /* Size of machine word in bytes. Must be power of 2. */ + /* Minimum number of bytes requested at a time from the system. Must be + * multiple of WORDSIZE. */ + + size_t remaining; /* Number of bytes left in current block of storage. */ + void *base; /* Pointer to base of current block of storage. */ + void *loc; /* Current location in block to next allocate memory. */ + + void internal_init() { + remaining = 0; + base = NULL; + usedMemory = 0; + wastedMemory = 0; + } + +public: + size_t usedMemory; + size_t wastedMemory; + + /** + Default constructor. Initializes a new pool. + */ + PooledAllocator() { internal_init(); } + + /** + * Destructor. Frees all the memory allocated in this pool. + */ + ~PooledAllocator() { free_all(); } + + /** Frees all allocated memory chunks */ + void free_all() { + while (base != NULL) { + void *prev = + *(static_cast(base)); /* Get pointer to prev block. */ + ::free(base); + base = prev; + } + internal_init(); + } + + /** + * Returns a pointer to a piece of new memory of the given size in bytes + * allocated from the pool. + */ + void *malloc(const size_t req_size) { + /* Round size up to a multiple of wordsize. The following expression + only works for WORDSIZE that is a power of 2, by masking last bits of + incremented size to zero. + */ + const size_t size = (req_size + (WORDSIZE - 1)) & ~(WORDSIZE - 1); + + /* Check whether a new block must be allocated. Note that the first word + of a block is reserved for a pointer to the previous block. + */ + if (size > remaining) { + + wastedMemory += remaining; + + /* Allocate new storage. */ + const size_t blocksize = + (size + sizeof(void *) + (WORDSIZE - 1) > BLOCKSIZE) + ? size + sizeof(void *) + (WORDSIZE - 1) + : BLOCKSIZE; + + // use the standard C malloc to allocate memory + void *m = ::malloc(blocksize); + if (!m) { + fprintf(stderr, "Failed to allocate memory.\n"); + return NULL; + } + + /* Fill first word of new block with pointer to previous block. */ + static_cast(m)[0] = base; + base = m; + + size_t shift = 0; + // int size_t = (WORDSIZE - ( (((size_t)m) + sizeof(void*)) & + // (WORDSIZE-1))) & (WORDSIZE-1); + + remaining = blocksize - sizeof(void *) - shift; + loc = (static_cast(m) + sizeof(void *) + shift); + } + void *rloc = loc; + loc = static_cast(loc) + size; + remaining -= size; + + usedMemory += size; + + return rloc; + } + + /** + * Allocates (using this pool) a generic type T. + * + * Params: + * count = number of instances to allocate. + * Returns: pointer (of type T*) to memory buffer + */ + template T *allocate(const size_t count = 1) { + T *mem = static_cast(this->malloc(sizeof(T) * count)); + return mem; + } +}; +/** @} */ + +/** @addtogroup nanoflann_metaprog_grp Auxiliary metaprogramming stuff + * @{ */ + +/** Used to declare fixed-size arrays when DIM>0, dynamically-allocated vectors + * when DIM=-1. Fixed size version for a generic DIM: + */ +template struct array_or_vector_selector { + typedef std::array container_t; +}; +/** Dynamic size version */ +template struct array_or_vector_selector<-1, T> { + typedef std::vector container_t; +}; + +/** @} */ + +/** kd-tree base-class + * + * Contains the member functions common to the classes KDTreeSingleIndexAdaptor + * and KDTreeSingleIndexDynamicAdaptor_. + * + * \tparam Derived The name of the class which inherits this class. + * \tparam DatasetAdaptor The user-provided adaptor (see comments above). + * \tparam Distance The distance metric to use, these are all classes derived + * from nanoflann::Metric \tparam DIM Dimensionality of data points (e.g. 3 for + * 3D points) \tparam IndexType Will be typically size_t or int + */ + +template +class KDTreeBaseClass { + +public: + /** Frees the previously-built index. Automatically called within + * buildIndex(). */ + void freeIndex(Derived &obj) { + obj.pool.free_all(); + obj.root_node = NULL; + obj.m_size_at_index_build = 0; + } + + typedef typename Distance::ElementType ElementType; + typedef typename Distance::DistanceType DistanceType; + + /*--------------------- Internal Data Structures --------------------------*/ + struct Node { + /** Union used because a node can be either a LEAF node or a non-leaf node, + * so both data fields are never used simultaneously */ + union { + struct leaf { + IndexType left, right; //!< Indices of points in leaf node + } lr; + struct nonleaf { + int divfeat; //!< Dimension used for subdivision. + DistanceType divlow, divhigh; //!< The values used for subdivision. + } sub; + } node_type; + Node *child1, *child2; //!< Child nodes (both=NULL mean its a leaf node) + }; + + typedef Node *NodePtr; + + struct Interval { + ElementType low, high; + }; + + /** + * Array of indices to vectors in the dataset. + */ + std::vector vind; + + NodePtr root_node; + + size_t m_leaf_max_size; + + size_t m_size; //!< Number of current points in the dataset + size_t m_size_at_index_build; //!< Number of points in the dataset when the + //!< index was built + int dim; //!< Dimensionality of each data point + + /** Define "BoundingBox" as a fixed-size or variable-size container depending + * on "DIM" */ + typedef + typename array_or_vector_selector::container_t BoundingBox; + + /** Define "distance_vector_t" as a fixed-size or variable-size container + * depending on "DIM" */ + typedef typename array_or_vector_selector::container_t + distance_vector_t; + + /** The KD-tree used to find neighbours */ + + BoundingBox root_bbox; + + /** + * Pooled memory allocator. + * + * Using a pooled memory allocator is more efficient + * than allocating memory directly when there is a large + * number small of memory allocations. + */ + PooledAllocator pool; + + /** Returns number of points in dataset */ + size_t size(const Derived &obj) const { return obj.m_size; } + + /** Returns the length of each point in the dataset */ + size_t veclen(const Derived &obj) { + return static_cast(DIM > 0 ? DIM : obj.dim); + } + + /// Helper accessor to the dataset points: + inline ElementType dataset_get(const Derived &obj, size_t idx, + int component) const { + return obj.dataset.kdtree_get_pt(idx, component); + } + + /** + * Computes the inde memory usage + * Returns: memory used by the index + */ + size_t usedMemory(Derived &obj) { + return obj.pool.usedMemory + obj.pool.wastedMemory + + obj.dataset.kdtree_get_point_count() * + sizeof(IndexType); // pool memory and vind array memory + } + + void computeMinMax(const Derived &obj, IndexType *ind, IndexType count, + int element, ElementType &min_elem, + ElementType &max_elem) { + min_elem = dataset_get(obj, ind[0], element); + max_elem = dataset_get(obj, ind[0], element); + for (IndexType i = 1; i < count; ++i) { + ElementType val = dataset_get(obj, ind[i], element); + if (val < min_elem) + min_elem = val; + if (val > max_elem) + max_elem = val; + } + } + + /** + * Create a tree node that subdivides the list of vecs from vind[first] + * to vind[last]. The routine is called recursively on each sublist. + * + * @param left index of the first vector + * @param right index of the last vector + */ + NodePtr divideTree(Derived &obj, const IndexType left, const IndexType right, + BoundingBox &bbox) { + NodePtr node = obj.pool.template allocate(); // allocate memory + + /* If too few exemplars remain, then make this a leaf node. */ + if ((right - left) <= static_cast(obj.m_leaf_max_size)) { + node->child1 = node->child2 = NULL; /* Mark as leaf node. */ + node->node_type.lr.left = left; + node->node_type.lr.right = right; + + // compute bounding-box of leaf points + for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) { + bbox[i].low = dataset_get(obj, obj.vind[left], i); + bbox[i].high = dataset_get(obj, obj.vind[left], i); + } + for (IndexType k = left + 1; k < right; ++k) { + for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) { + if (bbox[i].low > dataset_get(obj, obj.vind[k], i)) + bbox[i].low = dataset_get(obj, obj.vind[k], i); + if (bbox[i].high < dataset_get(obj, obj.vind[k], i)) + bbox[i].high = dataset_get(obj, obj.vind[k], i); + } + } + } else { + IndexType idx; + int cutfeat; + DistanceType cutval; + middleSplit_(obj, &obj.vind[0] + left, right - left, idx, cutfeat, cutval, + bbox); + + node->node_type.sub.divfeat = cutfeat; + + BoundingBox left_bbox(bbox); + left_bbox[cutfeat].high = cutval; + node->child1 = divideTree(obj, left, left + idx, left_bbox); + + BoundingBox right_bbox(bbox); + right_bbox[cutfeat].low = cutval; + node->child2 = divideTree(obj, left + idx, right, right_bbox); + + node->node_type.sub.divlow = left_bbox[cutfeat].high; + node->node_type.sub.divhigh = right_bbox[cutfeat].low; + + for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) { + bbox[i].low = std::min(left_bbox[i].low, right_bbox[i].low); + bbox[i].high = std::max(left_bbox[i].high, right_bbox[i].high); + } + } + + return node; + } + + void middleSplit_(Derived &obj, IndexType *ind, IndexType count, + IndexType &index, int &cutfeat, DistanceType &cutval, + const BoundingBox &bbox) { + const DistanceType EPS = static_cast(0.00001); + ElementType max_span = bbox[0].high - bbox[0].low; + for (int i = 1; i < (DIM > 0 ? DIM : obj.dim); ++i) { + ElementType span = bbox[i].high - bbox[i].low; + if (span > max_span) { + max_span = span; + } + } + ElementType max_spread = -1; + cutfeat = 0; + for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) { + ElementType span = bbox[i].high - bbox[i].low; + if (span > (1 - EPS) * max_span) { + ElementType min_elem, max_elem; + computeMinMax(obj, ind, count, i, min_elem, max_elem); + ElementType spread = max_elem - min_elem; + ; + if (spread > max_spread) { + cutfeat = i; + max_spread = spread; + } + } + } + // split in the middle + DistanceType split_val = (bbox[cutfeat].low + bbox[cutfeat].high) / 2; + ElementType min_elem, max_elem; + computeMinMax(obj, ind, count, cutfeat, min_elem, max_elem); + + if (split_val < min_elem) + cutval = min_elem; + else if (split_val > max_elem) + cutval = max_elem; + else + cutval = split_val; + + IndexType lim1, lim2; + planeSplit(obj, ind, count, cutfeat, cutval, lim1, lim2); + + if (lim1 > count / 2) + index = lim1; + else if (lim2 < count / 2) + index = lim2; + else + index = count / 2; + } + + /** + * Subdivide the list of points by a plane perpendicular on axe corresponding + * to the 'cutfeat' dimension at 'cutval' position. + * + * On return: + * dataset[ind[0..lim1-1]][cutfeat]cutval + */ + void planeSplit(Derived &obj, IndexType *ind, const IndexType count, + int cutfeat, DistanceType &cutval, IndexType &lim1, + IndexType &lim2) { + /* Move vector indices for left subtree to front of list. */ + IndexType left = 0; + IndexType right = count - 1; + for (;;) { + while (left <= right && dataset_get(obj, ind[left], cutfeat) < cutval) + ++left; + while (right && left <= right && + dataset_get(obj, ind[right], cutfeat) >= cutval) + --right; + if (left > right || !right) + break; // "!right" was added to support unsigned Index types + std::swap(ind[left], ind[right]); + ++left; + --right; + } + /* If either list is empty, it means that all remaining features + * are identical. Split in the middle to maintain a balanced tree. + */ + lim1 = left; + right = count - 1; + for (;;) { + while (left <= right && dataset_get(obj, ind[left], cutfeat) <= cutval) + ++left; + while (right && left <= right && + dataset_get(obj, ind[right], cutfeat) > cutval) + --right; + if (left > right || !right) + break; // "!right" was added to support unsigned Index types + std::swap(ind[left], ind[right]); + ++left; + --right; + } + lim2 = left; + } + + DistanceType computeInitialDistances(const Derived &obj, + const ElementType *vec, + distance_vector_t &dists) const { + assert(vec); + DistanceType distsq = DistanceType(); + + for (int i = 0; i < (DIM > 0 ? DIM : obj.dim); ++i) { + if (vec[i] < obj.root_bbox[i].low) { + dists[i] = obj.distance.accum_dist(vec[i], obj.root_bbox[i].low, i); + distsq += dists[i]; + } + if (vec[i] > obj.root_bbox[i].high) { + dists[i] = obj.distance.accum_dist(vec[i], obj.root_bbox[i].high, i); + distsq += dists[i]; + } + } + return distsq; + } + + void save_tree(Derived &obj, FILE *stream, NodePtr tree) { + save_value(stream, *tree); + if (tree->child1 != NULL) { + save_tree(obj, stream, tree->child1); + } + if (tree->child2 != NULL) { + save_tree(obj, stream, tree->child2); + } + } + + void load_tree(Derived &obj, FILE *stream, NodePtr &tree) { + tree = obj.pool.template allocate(); + load_value(stream, *tree); + if (tree->child1 != NULL) { + load_tree(obj, stream, tree->child1); + } + if (tree->child2 != NULL) { + load_tree(obj, stream, tree->child2); + } + } + + /** Stores the index in a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so when + * loading the index object it must be constructed associated to the same + * source of data points used while building it. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void saveIndex_(Derived &obj, FILE *stream) { + save_value(stream, obj.m_size); + save_value(stream, obj.dim); + save_value(stream, obj.root_bbox); + save_value(stream, obj.m_leaf_max_size); + save_value(stream, obj.vind); + save_tree(obj, stream, obj.root_node); + } + + /** Loads a previous index from a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so the + * index object must be constructed associated to the same source of data + * points used while building the index. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void loadIndex_(Derived &obj, FILE *stream) { + load_value(stream, obj.m_size); + load_value(stream, obj.dim); + load_value(stream, obj.root_bbox); + load_value(stream, obj.m_leaf_max_size); + load_value(stream, obj.vind); + load_tree(obj, stream, obj.root_node); + } +}; + +/** @addtogroup kdtrees_grp KD-tree classes and adaptors + * @{ */ + +/** kd-tree static index + * + * Contains the k-d trees and other information for indexing a set of points + * for nearest-neighbor matching. + * + * The class "DatasetAdaptor" must provide the following interface (can be + * non-virtual, inlined methods): + * + * \code + * // Must return the number of data poins + * inline size_t kdtree_get_point_count() const { ... } + * + * + * // Must return the dim'th component of the idx'th point in the class: + * inline T kdtree_get_pt(const size_t idx, const size_t dim) const { ... } + * + * // Optional bounding-box computation: return false to default to a standard + * bbox computation loop. + * // Return true if the BBOX was already computed by the class and returned + * in "bb" so it can be avoided to redo it again. + * // Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3 + * for point clouds) template bool kdtree_get_bbox(BBOX &bb) const + * { + * bb[0].low = ...; bb[0].high = ...; // 0th dimension limits + * bb[1].low = ...; bb[1].high = ...; // 1st dimension limits + * ... + * return true; + * } + * + * \endcode + * + * \tparam DatasetAdaptor The user-provided adaptor (see comments above). + * \tparam Distance The distance metric to use: nanoflann::metric_L1, + * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. \tparam DIM + * Dimensionality of data points (e.g. 3 for 3D points) \tparam IndexType Will + * be typically size_t or int + */ +template +class KDTreeSingleIndexAdaptor + : public KDTreeBaseClass< + KDTreeSingleIndexAdaptor, + Distance, DatasetAdaptor, DIM, IndexType> { +public: + /** Deleted copy constructor*/ + KDTreeSingleIndexAdaptor( + const KDTreeSingleIndexAdaptor + &) = delete; + + /** + * The dataset used by this index + */ + const DatasetAdaptor &dataset; //!< The source of our data + + const KDTreeSingleIndexAdaptorParams index_params; + + Distance distance; + + typedef typename nanoflann::KDTreeBaseClass< + nanoflann::KDTreeSingleIndexAdaptor, + Distance, DatasetAdaptor, DIM, IndexType> + BaseClassRef; + + typedef typename BaseClassRef::ElementType ElementType; + typedef typename BaseClassRef::DistanceType DistanceType; + + typedef typename BaseClassRef::Node Node; + typedef Node *NodePtr; + + typedef typename BaseClassRef::Interval Interval; + /** Define "BoundingBox" as a fixed-size or variable-size container depending + * on "DIM" */ + typedef typename BaseClassRef::BoundingBox BoundingBox; + + /** Define "distance_vector_t" as a fixed-size or variable-size container + * depending on "DIM" */ + typedef typename BaseClassRef::distance_vector_t distance_vector_t; + + /** + * KDTree constructor + * + * Refer to docs in README.md or online in + * https://github.com/jlblancoc/nanoflann + * + * The KD-Tree point dimension (the length of each point in the datase, e.g. 3 + * for 3D points) is determined by means of: + * - The \a DIM template parameter if >0 (highest priority) + * - Otherwise, the \a dimensionality parameter of this constructor. + * + * @param inputData Dataset with the input features + * @param params Basically, the maximum leaf node size + */ + KDTreeSingleIndexAdaptor(const int dimensionality, + const DatasetAdaptor &inputData, + const KDTreeSingleIndexAdaptorParams ¶ms = + KDTreeSingleIndexAdaptorParams()) + : dataset(inputData), index_params(params), distance(inputData) { + BaseClassRef::root_node = NULL; + BaseClassRef::m_size = dataset.kdtree_get_point_count(); + BaseClassRef::m_size_at_index_build = BaseClassRef::m_size; + BaseClassRef::dim = dimensionality; + if (DIM > 0) + BaseClassRef::dim = DIM; + BaseClassRef::m_leaf_max_size = params.leaf_max_size; + + // Create a permutable array of indices to the input vectors. + init_vind(); + } + + /** + * Builds the index + */ + void buildIndex() { + BaseClassRef::m_size = dataset.kdtree_get_point_count(); + BaseClassRef::m_size_at_index_build = BaseClassRef::m_size; + init_vind(); + this->freeIndex(*this); + BaseClassRef::m_size_at_index_build = BaseClassRef::m_size; + if (BaseClassRef::m_size == 0) + return; + computeBoundingBox(BaseClassRef::root_bbox); + BaseClassRef::root_node = + this->divideTree(*this, 0, BaseClassRef::m_size, + BaseClassRef::root_bbox); // construct the tree + } + + /** \name Query methods + * @{ */ + + /** + * Find set of nearest neighbors to vec[0:dim-1]. Their indices are stored + * inside the result object. + * + * Params: + * result = the result object in which the indices of the + * nearest-neighbors are stored vec = the vector for which to search the + * nearest neighbors + * + * \tparam RESULTSET Should be any ResultSet + * \return True if the requested neighbors could be found. + * \sa knnSearch, radiusSearch + */ + template + bool findNeighbors(RESULTSET &result, const ElementType *vec, + const SearchParams &searchParams) const { + assert(vec); + if (this->size(*this) == 0) + return false; + if (!BaseClassRef::root_node) + throw std::runtime_error( + "[nanoflann] findNeighbors() called before building the index."); + float epsError = 1 + searchParams.eps; + + distance_vector_t + dists; // fixed or variable-sized container (depending on DIM) + auto zero = static_cast(0); + assign(dists, (DIM > 0 ? DIM : BaseClassRef::dim), + zero); // Fill it with zeros. + DistanceType distsq = this->computeInitialDistances(*this, vec, dists); + searchLevel(result, vec, BaseClassRef::root_node, distsq, dists, + epsError); // "count_leaf" parameter removed since was neither + // used nor returned to the user. + return result.full(); + } + + /** + * Find the "num_closest" nearest neighbors to the \a query_point[0:dim-1]. + * Their indices are stored inside the result object. \sa radiusSearch, + * findNeighbors \note nChecks_IGNORED is ignored but kept for compatibility + * with the original FLANN interface. \return Number `N` of valid points in + * the result set. Only the first `N` entries in `out_indices` and + * `out_distances_sq` will be valid. Return may be less than `num_closest` + * only if the number of elements in the tree is less than `num_closest`. + */ + size_t knnSearch(const ElementType *query_point, const size_t num_closest, + IndexType *out_indices, DistanceType *out_distances_sq, + const int /* nChecks_IGNORED */ = 10) const { + nanoflann::KNNResultSet resultSet(num_closest); + resultSet.init(out_indices, out_distances_sq); + this->findNeighbors(resultSet, query_point, nanoflann::SearchParams()); + return resultSet.size(); + } + + /** + * Find all the neighbors to \a query_point[0:dim-1] within a maximum radius. + * The output is given as a vector of pairs, of which the first element is a + * point index and the second the corresponding distance. Previous contents of + * \a IndicesDists are cleared. + * + * If searchParams.sorted==true, the output list is sorted by ascending + * distances. + * + * For a better performance, it is advisable to do a .reserve() on the vector + * if you have any wild guess about the number of expected matches. + * + * \sa knnSearch, findNeighbors, radiusSearchCustomCallback + * \return The number of points within the given radius (i.e. indices.size() + * or dists.size() ) + */ + size_t + radiusSearch(const ElementType *query_point, const DistanceType &radius, + std::vector> &IndicesDists, + const SearchParams &searchParams) const { + RadiusResultSet resultSet(radius, IndicesDists); + const size_t nFound = + radiusSearchCustomCallback(query_point, resultSet, searchParams); + if (searchParams.sorted) + std::sort(IndicesDists.begin(), IndicesDists.end(), IndexDist_Sorter()); + return nFound; + } + + /** + * Just like radiusSearch() but with a custom callback class for each point + * found in the radius of the query. See the source of RadiusResultSet<> as a + * start point for your own classes. \sa radiusSearch + */ + template + size_t radiusSearchCustomCallback( + const ElementType *query_point, SEARCH_CALLBACK &resultSet, + const SearchParams &searchParams = SearchParams()) const { + this->findNeighbors(resultSet, query_point, searchParams); + return resultSet.size(); + } + + /** @} */ + +public: + /** Make sure the auxiliary list \a vind has the same size than the current + * dataset, and re-generate if size has changed. */ + void init_vind() { + // Create a permutable array of indices to the input vectors. + BaseClassRef::m_size = dataset.kdtree_get_point_count(); + if (BaseClassRef::vind.size() != BaseClassRef::m_size) + BaseClassRef::vind.resize(BaseClassRef::m_size); + for (size_t i = 0; i < BaseClassRef::m_size; i++) + BaseClassRef::vind[i] = i; + } + + void computeBoundingBox(BoundingBox &bbox) { + resize(bbox, (DIM > 0 ? DIM : BaseClassRef::dim)); + if (dataset.kdtree_get_bbox(bbox)) { + // Done! It was implemented in derived class + } else { + const size_t N = dataset.kdtree_get_point_count(); + if (!N) + throw std::runtime_error("[nanoflann] computeBoundingBox() called but " + "no data points found."); + for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) { + bbox[i].low = bbox[i].high = this->dataset_get(*this, 0, i); + } + for (size_t k = 1; k < N; ++k) { + for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) { + if (this->dataset_get(*this, k, i) < bbox[i].low) + bbox[i].low = this->dataset_get(*this, k, i); + if (this->dataset_get(*this, k, i) > bbox[i].high) + bbox[i].high = this->dataset_get(*this, k, i); + } + } + } + } + + /** + * Performs an exact search in the tree starting from a node. + * \tparam RESULTSET Should be any ResultSet + * \return true if the search should be continued, false if the results are + * sufficient + */ + template + bool searchLevel(RESULTSET &result_set, const ElementType *vec, + const NodePtr node, DistanceType mindistsq, + distance_vector_t &dists, const float epsError) const { + /* If this is a leaf node, then do check and return. */ + if ((node->child1 == NULL) && (node->child2 == NULL)) { + // count_leaf += (node->lr.right-node->lr.left); // Removed since was + // neither used nor returned to the user. + DistanceType worst_dist = result_set.worstDist(); + for (IndexType i = node->node_type.lr.left; i < node->node_type.lr.right; + ++i) { + const IndexType index = BaseClassRef::vind[i]; // reorder... : i; + DistanceType dist = distance.evalMetric( + vec, index, (DIM > 0 ? DIM : BaseClassRef::dim)); + if (dist < worst_dist) { + if (!result_set.addPoint(dist, BaseClassRef::vind[i])) { + // the resultset doesn't want to receive any more points, we're done + // searching! + return false; + } + } + } + return true; + } + + /* Which child branch should be taken first? */ + int idx = node->node_type.sub.divfeat; + ElementType val = vec[idx]; + DistanceType diff1 = val - node->node_type.sub.divlow; + DistanceType diff2 = val - node->node_type.sub.divhigh; + + NodePtr bestChild; + NodePtr otherChild; + DistanceType cut_dist; + if ((diff1 + diff2) < 0) { + bestChild = node->child1; + otherChild = node->child2; + cut_dist = distance.accum_dist(val, node->node_type.sub.divhigh, idx); + } else { + bestChild = node->child2; + otherChild = node->child1; + cut_dist = distance.accum_dist(val, node->node_type.sub.divlow, idx); + } + + /* Call recursively to search next level down. */ + if (!searchLevel(result_set, vec, bestChild, mindistsq, dists, epsError)) { + // the resultset doesn't want to receive any more points, we're done + // searching! + return false; + } + + DistanceType dst = dists[idx]; + mindistsq = mindistsq + cut_dist - dst; + dists[idx] = cut_dist; + if (mindistsq * epsError <= result_set.worstDist()) { + if (!searchLevel(result_set, vec, otherChild, mindistsq, dists, + epsError)) { + // the resultset doesn't want to receive any more points, we're done + // searching! + return false; + } + } + dists[idx] = dst; + return true; + } + +public: + /** Stores the index in a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so when + * loading the index object it must be constructed associated to the same + * source of data points used while building it. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void saveIndex(FILE *stream) { this->saveIndex_(*this, stream); } + + /** Loads a previous index from a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so the + * index object must be constructed associated to the same source of data + * points used while building the index. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void loadIndex(FILE *stream) { this->loadIndex_(*this, stream); } + +}; // class KDTree + +/** kd-tree dynamic index + * + * Contains the k-d trees and other information for indexing a set of points + * for nearest-neighbor matching. + * + * The class "DatasetAdaptor" must provide the following interface (can be + * non-virtual, inlined methods): + * + * \code + * // Must return the number of data poins + * inline size_t kdtree_get_point_count() const { ... } + * + * // Must return the dim'th component of the idx'th point in the class: + * inline T kdtree_get_pt(const size_t idx, const size_t dim) const { ... } + * + * // Optional bounding-box computation: return false to default to a standard + * bbox computation loop. + * // Return true if the BBOX was already computed by the class and returned + * in "bb" so it can be avoided to redo it again. + * // Look at bb.size() to find out the expected dimensionality (e.g. 2 or 3 + * for point clouds) template bool kdtree_get_bbox(BBOX &bb) const + * { + * bb[0].low = ...; bb[0].high = ...; // 0th dimension limits + * bb[1].low = ...; bb[1].high = ...; // 1st dimension limits + * ... + * return true; + * } + * + * \endcode + * + * \tparam DatasetAdaptor The user-provided adaptor (see comments above). + * \tparam Distance The distance metric to use: nanoflann::metric_L1, + * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. \tparam DIM + * Dimensionality of data points (e.g. 3 for 3D points) \tparam IndexType Will + * be typically size_t or int + */ +template +class KDTreeSingleIndexDynamicAdaptor_ + : public KDTreeBaseClass, + Distance, DatasetAdaptor, DIM, IndexType> { +public: + /** + * The dataset used by this index + */ + const DatasetAdaptor &dataset; //!< The source of our data + + KDTreeSingleIndexAdaptorParams index_params; + + std::vector &treeIndex; + + Distance distance; + + typedef typename nanoflann::KDTreeBaseClass< + nanoflann::KDTreeSingleIndexDynamicAdaptor_, + Distance, DatasetAdaptor, DIM, IndexType> + BaseClassRef; + + typedef typename BaseClassRef::ElementType ElementType; + typedef typename BaseClassRef::DistanceType DistanceType; + + typedef typename BaseClassRef::Node Node; + typedef Node *NodePtr; + + typedef typename BaseClassRef::Interval Interval; + /** Define "BoundingBox" as a fixed-size or variable-size container depending + * on "DIM" */ + typedef typename BaseClassRef::BoundingBox BoundingBox; + + /** Define "distance_vector_t" as a fixed-size or variable-size container + * depending on "DIM" */ + typedef typename BaseClassRef::distance_vector_t distance_vector_t; + + /** + * KDTree constructor + * + * Refer to docs in README.md or online in + * https://github.com/jlblancoc/nanoflann + * + * The KD-Tree point dimension (the length of each point in the datase, e.g. 3 + * for 3D points) is determined by means of: + * - The \a DIM template parameter if >0 (highest priority) + * - Otherwise, the \a dimensionality parameter of this constructor. + * + * @param inputData Dataset with the input features + * @param params Basically, the maximum leaf node size + */ + KDTreeSingleIndexDynamicAdaptor_( + const int dimensionality, const DatasetAdaptor &inputData, + std::vector &treeIndex_, + const KDTreeSingleIndexAdaptorParams ¶ms = + KDTreeSingleIndexAdaptorParams()) + : dataset(inputData), index_params(params), treeIndex(treeIndex_), + distance(inputData) { + BaseClassRef::root_node = NULL; + BaseClassRef::m_size = 0; + BaseClassRef::m_size_at_index_build = 0; + BaseClassRef::dim = dimensionality; + if (DIM > 0) + BaseClassRef::dim = DIM; + BaseClassRef::m_leaf_max_size = params.leaf_max_size; + } + + /** Assignment operator definiton */ + KDTreeSingleIndexDynamicAdaptor_ + operator=(const KDTreeSingleIndexDynamicAdaptor_ &rhs) { + KDTreeSingleIndexDynamicAdaptor_ tmp(rhs); + std::swap(BaseClassRef::vind, tmp.BaseClassRef::vind); + std::swap(BaseClassRef::m_leaf_max_size, tmp.BaseClassRef::m_leaf_max_size); + std::swap(index_params, tmp.index_params); + std::swap(treeIndex, tmp.treeIndex); + std::swap(BaseClassRef::m_size, tmp.BaseClassRef::m_size); + std::swap(BaseClassRef::m_size_at_index_build, + tmp.BaseClassRef::m_size_at_index_build); + std::swap(BaseClassRef::root_node, tmp.BaseClassRef::root_node); + std::swap(BaseClassRef::root_bbox, tmp.BaseClassRef::root_bbox); + std::swap(BaseClassRef::pool, tmp.BaseClassRef::pool); + return *this; + } + + /** + * Builds the index + */ + void buildIndex() { + BaseClassRef::m_size = BaseClassRef::vind.size(); + this->freeIndex(*this); + BaseClassRef::m_size_at_index_build = BaseClassRef::m_size; + if (BaseClassRef::m_size == 0) + return; + computeBoundingBox(BaseClassRef::root_bbox); + BaseClassRef::root_node = + this->divideTree(*this, 0, BaseClassRef::m_size, + BaseClassRef::root_bbox); // construct the tree + } + + /** \name Query methods + * @{ */ + + /** + * Find set of nearest neighbors to vec[0:dim-1]. Their indices are stored + * inside the result object. + * + * Params: + * result = the result object in which the indices of the + * nearest-neighbors are stored vec = the vector for which to search the + * nearest neighbors + * + * \tparam RESULTSET Should be any ResultSet + * \return True if the requested neighbors could be found. + * \sa knnSearch, radiusSearch + */ + template + bool findNeighbors(RESULTSET &result, const ElementType *vec, + const SearchParams &searchParams) const { + assert(vec); + if (this->size(*this) == 0) + return false; + if (!BaseClassRef::root_node) + return false; + float epsError = 1 + searchParams.eps; + + // fixed or variable-sized container (depending on DIM) + distance_vector_t dists; + // Fill it with zeros. + assign(dists, (DIM > 0 ? DIM : BaseClassRef::dim), + static_cast(0)); + DistanceType distsq = this->computeInitialDistances(*this, vec, dists); + searchLevel(result, vec, BaseClassRef::root_node, distsq, dists, + epsError); // "count_leaf" parameter removed since was neither + // used nor returned to the user. + return result.full(); + } + + /** + * Find the "num_closest" nearest neighbors to the \a query_point[0:dim-1]. + * Their indices are stored inside the result object. \sa radiusSearch, + * findNeighbors \note nChecks_IGNORED is ignored but kept for compatibility + * with the original FLANN interface. \return Number `N` of valid points in + * the result set. Only the first `N` entries in `out_indices` and + * `out_distances_sq` will be valid. Return may be less than `num_closest` + * only if the number of elements in the tree is less than `num_closest`. + */ + size_t knnSearch(const ElementType *query_point, const size_t num_closest, + IndexType *out_indices, DistanceType *out_distances_sq, + const int /* nChecks_IGNORED */ = 10) const { + nanoflann::KNNResultSet resultSet(num_closest); + resultSet.init(out_indices, out_distances_sq); + this->findNeighbors(resultSet, query_point, nanoflann::SearchParams()); + return resultSet.size(); + } + + /** + * Find all the neighbors to \a query_point[0:dim-1] within a maximum radius. + * The output is given as a vector of pairs, of which the first element is a + * point index and the second the corresponding distance. Previous contents of + * \a IndicesDists are cleared. + * + * If searchParams.sorted==true, the output list is sorted by ascending + * distances. + * + * For a better performance, it is advisable to do a .reserve() on the vector + * if you have any wild guess about the number of expected matches. + * + * \sa knnSearch, findNeighbors, radiusSearchCustomCallback + * \return The number of points within the given radius (i.e. indices.size() + * or dists.size() ) + */ + size_t + radiusSearch(const ElementType *query_point, const DistanceType &radius, + std::vector> &IndicesDists, + const SearchParams &searchParams) const { + RadiusResultSet resultSet(radius, IndicesDists); + const size_t nFound = + radiusSearchCustomCallback(query_point, resultSet, searchParams); + if (searchParams.sorted) + std::sort(IndicesDists.begin(), IndicesDists.end(), IndexDist_Sorter()); + return nFound; + } + + /** + * Just like radiusSearch() but with a custom callback class for each point + * found in the radius of the query. See the source of RadiusResultSet<> as a + * start point for your own classes. \sa radiusSearch + */ + template + size_t radiusSearchCustomCallback( + const ElementType *query_point, SEARCH_CALLBACK &resultSet, + const SearchParams &searchParams = SearchParams()) const { + this->findNeighbors(resultSet, query_point, searchParams); + return resultSet.size(); + } + + /** @} */ + +public: + void computeBoundingBox(BoundingBox &bbox) { + resize(bbox, (DIM > 0 ? DIM : BaseClassRef::dim)); + + if (dataset.kdtree_get_bbox(bbox)) { + // Done! It was implemented in derived class + } else { + const size_t N = BaseClassRef::m_size; + if (!N) + throw std::runtime_error("[nanoflann] computeBoundingBox() called but " + "no data points found."); + for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) { + bbox[i].low = bbox[i].high = + this->dataset_get(*this, BaseClassRef::vind[0], i); + } + for (size_t k = 1; k < N; ++k) { + for (int i = 0; i < (DIM > 0 ? DIM : BaseClassRef::dim); ++i) { + if (this->dataset_get(*this, BaseClassRef::vind[k], i) < bbox[i].low) + bbox[i].low = this->dataset_get(*this, BaseClassRef::vind[k], i); + if (this->dataset_get(*this, BaseClassRef::vind[k], i) > bbox[i].high) + bbox[i].high = this->dataset_get(*this, BaseClassRef::vind[k], i); + } + } + } + } + + /** + * Performs an exact search in the tree starting from a node. + * \tparam RESULTSET Should be any ResultSet + */ + template + void searchLevel(RESULTSET &result_set, const ElementType *vec, + const NodePtr node, DistanceType mindistsq, + distance_vector_t &dists, const float epsError) const { + /* If this is a leaf node, then do check and return. */ + if ((node->child1 == NULL) && (node->child2 == NULL)) { + // count_leaf += (node->lr.right-node->lr.left); // Removed since was + // neither used nor returned to the user. + DistanceType worst_dist = result_set.worstDist(); + for (IndexType i = node->node_type.lr.left; i < node->node_type.lr.right; + ++i) { + const IndexType index = BaseClassRef::vind[i]; // reorder... : i; + if (treeIndex[index] == -1) + continue; + DistanceType dist = distance.evalMetric( + vec, index, (DIM > 0 ? DIM : BaseClassRef::dim)); + if (dist < worst_dist) { + if (!result_set.addPoint( + static_cast(dist), + static_cast( + BaseClassRef::vind[i]))) { + // the resultset doesn't want to receive any more points, we're done + // searching! + return; // false; + } + } + } + return; + } + + /* Which child branch should be taken first? */ + int idx = node->node_type.sub.divfeat; + ElementType val = vec[idx]; + DistanceType diff1 = val - node->node_type.sub.divlow; + DistanceType diff2 = val - node->node_type.sub.divhigh; + + NodePtr bestChild; + NodePtr otherChild; + DistanceType cut_dist; + if ((diff1 + diff2) < 0) { + bestChild = node->child1; + otherChild = node->child2; + cut_dist = distance.accum_dist(val, node->node_type.sub.divhigh, idx); + } else { + bestChild = node->child2; + otherChild = node->child1; + cut_dist = distance.accum_dist(val, node->node_type.sub.divlow, idx); + } + + /* Call recursively to search next level down. */ + searchLevel(result_set, vec, bestChild, mindistsq, dists, epsError); + + DistanceType dst = dists[idx]; + mindistsq = mindistsq + cut_dist - dst; + dists[idx] = cut_dist; + if (mindistsq * epsError <= result_set.worstDist()) { + searchLevel(result_set, vec, otherChild, mindistsq, dists, epsError); + } + dists[idx] = dst; + } + +public: + /** Stores the index in a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so when + * loading the index object it must be constructed associated to the same + * source of data points used while building it. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void saveIndex(FILE *stream) { this->saveIndex_(*this, stream); } + + /** Loads a previous index from a binary file. + * IMPORTANT NOTE: The set of data points is NOT stored in the file, so the + * index object must be constructed associated to the same source of data + * points used while building the index. See the example: + * examples/saveload_example.cpp \sa loadIndex */ + void loadIndex(FILE *stream) { this->loadIndex_(*this, stream); } +}; + +/** kd-tree dynaimic index + * + * class to create multiple static index and merge their results to behave as + * single dynamic index as proposed in Logarithmic Approach. + * + * Example of usage: + * examples/dynamic_pointcloud_example.cpp + * + * \tparam DatasetAdaptor The user-provided adaptor (see comments above). + * \tparam Distance The distance metric to use: nanoflann::metric_L1, + * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. \tparam DIM + * Dimensionality of data points (e.g. 3 for 3D points) \tparam IndexType Will + * be typically size_t or int + */ +template +class KDTreeSingleIndexDynamicAdaptor { +public: + typedef typename Distance::ElementType ElementType; + typedef typename Distance::DistanceType DistanceType; + +protected: + size_t m_leaf_max_size; + size_t treeCount; + size_t pointCount; + + /** + * The dataset used by this index + */ + const DatasetAdaptor &dataset; //!< The source of our data + + std::vector treeIndex; //!< treeIndex[idx] is the index of tree in which + //!< point at idx is stored. treeIndex[idx]=-1 + //!< means that point has been removed. + + KDTreeSingleIndexAdaptorParams index_params; + + int dim; //!< Dimensionality of each data point + + typedef KDTreeSingleIndexDynamicAdaptor_ + index_container_t; + std::vector index; + +public: + /** Get a const ref to the internal list of indices; the number of indices is + * adapted dynamically as the dataset grows in size. */ + const std::vector &getAllIndices() const { return index; } + +private: + /** finds position of least significant unset bit */ + int First0Bit(IndexType num) { + int pos = 0; + while (num & 1) { + num = num >> 1; + pos++; + } + return pos; + } + + /** Creates multiple empty trees to handle dynamic support */ + void init() { + typedef KDTreeSingleIndexDynamicAdaptor_ + my_kd_tree_t; + std::vector index_( + treeCount, my_kd_tree_t(dim /*dim*/, dataset, treeIndex, index_params)); + index = index_; + } + +public: + Distance distance; + + /** + * KDTree constructor + * + * Refer to docs in README.md or online in + * https://github.com/jlblancoc/nanoflann + * + * The KD-Tree point dimension (the length of each point in the datase, e.g. 3 + * for 3D points) is determined by means of: + * - The \a DIM template parameter if >0 (highest priority) + * - Otherwise, the \a dimensionality parameter of this constructor. + * + * @param inputData Dataset with the input features + * @param params Basically, the maximum leaf node size + */ + KDTreeSingleIndexDynamicAdaptor(const int dimensionality, + const DatasetAdaptor &inputData, + const KDTreeSingleIndexAdaptorParams ¶ms = + KDTreeSingleIndexAdaptorParams(), + const size_t maximumPointCount = 1000000000U) + : dataset(inputData), index_params(params), distance(inputData) { + treeCount = static_cast(std::log2(maximumPointCount)); + pointCount = 0U; + dim = dimensionality; + treeIndex.clear(); + if (DIM > 0) + dim = DIM; + m_leaf_max_size = params.leaf_max_size; + init(); + const size_t num_initial_points = dataset.kdtree_get_point_count(); + if (num_initial_points > 0) { + addPoints(0, num_initial_points - 1); + } + } + + /** Deleted copy constructor*/ + KDTreeSingleIndexDynamicAdaptor( + const KDTreeSingleIndexDynamicAdaptor &) = delete; + + /** Add points to the set, Inserts all points from [start, end] */ + void addPoints(IndexType start, IndexType end) { + size_t count = end - start + 1; + treeIndex.resize(treeIndex.size() + count); + for (IndexType idx = start; idx <= end; idx++) { + int pos = First0Bit(pointCount); + index[pos].vind.clear(); + treeIndex[pointCount] = pos; + for (int i = 0; i < pos; i++) { + for (int j = 0; j < static_cast(index[i].vind.size()); j++) { + index[pos].vind.push_back(index[i].vind[j]); + if (treeIndex[index[i].vind[j]] != -1) + treeIndex[index[i].vind[j]] = pos; + } + index[i].vind.clear(); + index[i].freeIndex(index[i]); + } + index[pos].vind.push_back(idx); + index[pos].buildIndex(); + pointCount++; + } + } + + /** Remove a point from the set (Lazy Deletion) */ + void removePoint(size_t idx) { + if (idx >= pointCount) + return; + treeIndex[idx] = -1; + } + + /** + * Find set of nearest neighbors to vec[0:dim-1]. Their indices are stored + * inside the result object. + * + * Params: + * result = the result object in which the indices of the + * nearest-neighbors are stored vec = the vector for which to search the + * nearest neighbors + * + * \tparam RESULTSET Should be any ResultSet + * \return True if the requested neighbors could be found. + * \sa knnSearch, radiusSearch + */ + template + bool findNeighbors(RESULTSET &result, const ElementType *vec, + const SearchParams &searchParams) const { + for (size_t i = 0; i < treeCount; i++) { + index[i].findNeighbors(result, &vec[0], searchParams); + } + return result.full(); + } +}; + +/** An L2-metric KD-tree adaptor for working with data directly stored in an + * Eigen Matrix, without duplicating the data storage. Each row in the matrix + * represents a point in the state space. + * + * Example of usage: + * \code + * Eigen::Matrix mat; + * // Fill out "mat"... + * + * typedef KDTreeEigenMatrixAdaptor< Eigen::Matrix > + * my_kd_tree_t; const int max_leaf = 10; my_kd_tree_t mat_index(mat, max_leaf + * ); mat_index.index->buildIndex(); mat_index.index->... \endcode + * + * \tparam DIM If set to >0, it specifies a compile-time fixed dimensionality + * for the points in the data set, allowing more compiler optimizations. \tparam + * Distance The distance metric to use: nanoflann::metric_L1, + * nanoflann::metric_L2, nanoflann::metric_L2_Simple, etc. + */ +template +struct KDTreeEigenMatrixAdaptor { + typedef KDTreeEigenMatrixAdaptor self_t; + typedef typename MatrixType::Scalar num_t; + typedef typename MatrixType::Index IndexType; + typedef + typename Distance::template traits::distance_t metric_t; + typedef KDTreeSingleIndexAdaptor + index_t; + + index_t *index; //! The kd-tree index for the user to call its methods as + //! usual with any other FLANN index. + + /// Constructor: takes a const ref to the matrix object with the data points + KDTreeEigenMatrixAdaptor(const size_t dimensionality, + const std::reference_wrapper &mat, + const int leaf_max_size = 10) + : m_data_matrix(mat) { + const auto dims = mat.get().cols(); + if (size_t(dims) != dimensionality) + throw std::runtime_error( + "Error: 'dimensionality' must match column count in data matrix"); + if (DIM > 0 && int(dims) != DIM) + throw std::runtime_error( + "Data set dimensionality does not match the 'DIM' template argument"); + index = + new index_t(static_cast(dims), *this /* adaptor */, + nanoflann::KDTreeSingleIndexAdaptorParams(leaf_max_size)); + index->buildIndex(); + } + +public: + /** Deleted copy constructor */ + KDTreeEigenMatrixAdaptor(const self_t &) = delete; + + ~KDTreeEigenMatrixAdaptor() { delete index; } + + const std::reference_wrapper m_data_matrix; + + /** Query for the \a num_closest closest points to a given point (entered as + * query_point[0:dim-1]). Note that this is a short-cut method for + * index->findNeighbors(). The user can also call index->... methods as + * desired. \note nChecks_IGNORED is ignored but kept for compatibility with + * the original FLANN interface. + */ + inline void query(const num_t *query_point, const size_t num_closest, + IndexType *out_indices, num_t *out_distances_sq, + const int /* nChecks_IGNORED */ = 10) const { + nanoflann::KNNResultSet resultSet(num_closest); + resultSet.init(out_indices, out_distances_sq); + index->findNeighbors(resultSet, query_point, nanoflann::SearchParams()); + } + + /** @name Interface expected by KDTreeSingleIndexAdaptor + * @{ */ + + const self_t &derived() const { return *this; } + self_t &derived() { return *this; } + + // Must return the number of data points + inline size_t kdtree_get_point_count() const { + return m_data_matrix.get().rows(); + } + + // Returns the dim'th component of the idx'th point in the class: + inline num_t kdtree_get_pt(const IndexType idx, size_t dim) const { + return m_data_matrix.get().coeff(idx, IndexType(dim)); + } + + // Optional bounding-box computation: return false to default to a standard + // bbox computation loop. + // Return true if the BBOX was already computed by the class and returned in + // "bb" so it can be avoided to redo it again. Look at bb.size() to find out + // the expected dimensionality (e.g. 2 or 3 for point clouds) + template bool kdtree_get_bbox(BBOX & /*bb*/) const { + return false; + } + + /** @} */ + +}; // end of KDTreeEigenMatrixAdaptor + /** @} */ + +/** @} */ // end of grouping +} // namespace nanoflann + +#endif /* NANOFLANN_HPP_ */ diff --git a/modules/alphamat/src/precomp.hpp b/modules/alphamat/src/precomp.hpp new file mode 100644 index 00000000000..65ba4e7a21a --- /dev/null +++ b/modules/alphamat/src/precomp.hpp @@ -0,0 +1,19 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +#ifndef __OPENCV_PRECOMP_H__ +#define __OPENCV_PRECOMP_H__ + +#include +#include +#include + +#include "KtoU.hpp" +#include "intraU.hpp" +#include "cm.hpp" +#include "local_info.hpp" +#include "Eigen/IterativeLinearSolvers" +#include "trimming.hpp" + +#endif diff --git a/modules/alphamat/src/trimming.hpp b/modules/alphamat/src/trimming.hpp new file mode 100644 index 00000000000..106fca4a99a --- /dev/null +++ b/modules/alphamat/src/trimming.hpp @@ -0,0 +1,316 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. + +#include +#include +#include + +#include +#include "nanoflann.hpp" +#include "KDTreeVectorOfVectorsAdaptor.h" +using namespace nanoflann; +using namespace std; +using namespace cv; + + +typedef vector> my_vector_of_vectors_t; +typedef vector>> my_vector_of_set_t; +typedef vector my_vector_of_Mat; +typedef vector> my_vector_of_pair; + +my_vector_of_vectors_t fv_unk, fv_fg, fv_bg; +my_vector_of_Mat unkmean, fgmean, bgmean, unkcov, fgcov, bgcov; + +// void type2str(int type) { +// string r; + +// uchar depth = type & CV_MAT_DEPTH_MASK; +// uchar chans = 1 + (type >> CV_CN_SHIFT); + +// switch ( depth ) { +// case CV_8U: r = "8U"; break; +// case CV_8S: r = "8S"; break; +// case CV_16U: r = "16U"; break; +// case CV_16S: r = "16S"; break; +// case CV_32S: r = "32S"; break; +// case CV_32F: r = "32F"; break; +// case CV_64F: r = "64F"; break; +// default: r = "User"; break; +// } + +// r += "C"; +// r += (chans+'0'); +// cout<(i, j); + if (pix == 128) + unk++; + else if (pix > 200) + fg++; + else + bg++; + } + + fv_fg.resize(fg); fgmean.resize(fg); fgcov.resize(fg); + fv_bg.resize(bg); bgmean.resize(bg); bgcov.resize(bg); + fv_unk.resize(unk); unkmean.resize(unk); unkcov.resize(unk); map.resize(unk); + + for (i = win_size; i < img.rows-win_size; i++){ + for (j = win_size; j < img.cols-win_size; j++){ + float pix = tmap.at(i, j); + + // extract the window out of image + Mat win = img.rowRange(i-win_size, i+win_size+1); + win = win.colRange(j-win_size, j+win_size+1); + Mat win_ravel = Mat::zeros(9, 3, CV_64F); // doubt ?? + double sum1 = 0; + double sum2 = 0; + double sum3 = 0; + + int c = 0; + for (int p = 0; p < win_size*2+1; p++){ + for (int q = 0; q < win_size*2+1; q++){ + win_ravel.at(c, 0) = win.at(p,q)[0]/255.0; + win_ravel.at(c, 1) = win.at(p,q)[1]/255.0; + win_ravel.at(c, 2) = win.at(p,q)[2]/255.0; + + sum1 += win.at(p, q)[0]/255.0; + sum2 += win.at(p, q)[1]/255.0; + sum3 += win.at(p, q)[2]/255.0; + c++; + } + } + win = win_ravel; + + Mat win_mean = Mat::zeros(1, 3, CV_64F); + win_mean.at(0, 0) = sum1/num_win; + win_mean.at(0, 1) = sum2/num_win; + win_mean.at(0, 2) = sum3/num_win; + + // calculate the covariance matrix + Mat covariance = (win.t() * win / num_win) - (win_mean.t() * win_mean); + + if(pix == 128){ + fv_unk[c1].resize(3); + fv_unk[c1][0] = sum1/num_win; + fv_unk[c1][1] = sum2/num_win; + fv_unk[c1][2] = sum3/num_win; + unkmean[c1] = win_mean; + unkcov[c1] = covariance; + map[c1] = {i, j}; + if (c1 == 0){ + cout << i << " " << j << endl << endl; + } + c1++; + }else if(pix < 10){ + fv_bg[c2].resize(3); + fv_bg[c2][0] = sum1/num_win; + fv_bg[c2][1] = sum2/num_win; + fv_bg[c2][2] = sum3/num_win; + bgmean[c2] = win_mean; + bgcov[c2] = covariance; + c2++; + }else{ + fv_fg[c3].resize(3); + fv_fg[c3][0] = sum1/num_win; + fv_fg[c3][1] = sum2/num_win; + fv_fg[c3][2] = sum3/num_win; + fgmean[c3] = win_mean; + fgcov[c3] = covariance; + c3++; + } + //Bhattacharya distance + } + } +} + +void findNearestNbr(my_vector_of_vectors_t& indm){ + typedef KDTreeVectorOfVectorsAdaptor< my_vector_of_vectors_t, double > my_kd_tree_t; + my_kd_tree_t mat_index_fg(3 /*dim*/, fv_fg, 10 /* max leaf */ ); + mat_index_fg.index->buildIndex(); + + my_kd_tree_t mat_index_bg(3 /*dim*/, fv_bg, 10 /* max leaf */ ); + mat_index_bg.index->buildIndex(); + + // do a knn search 20 nbrs + const size_t num_results = 20; + + int N = fv_unk.size(); + + vector ret_indexes(num_results); + vector out_dists_sqr(num_results); + nanoflann::KNNResultSet resultSet(num_results); + + indm.resize(N); + int i = 0; + for (i = 0; i < fv_unk.size(); i++){ + indm[i].resize(2*num_results); + + resultSet.init(&ret_indexes[0], &out_dists_sqr[0] ); + mat_index_fg.index->findNeighbors(resultSet, &fv_unk[i][0], nanoflann::SearchParams(10)); + for (int j = 0; j < num_results; j++){ + // cout << "$$$$$$$ret_index["< tauf){ + new_tmap.at(imgi, imgj) = 255; // fg + cout << "fg" << endl; + }else if (minbg < tauc && minfg > tauf){ + new_tmap.at(imgi, imgj) = 0; // bg + cout << "bg" << endl; + } + // else remain unknown + } + + imwrite("1.png", tmap); + imwrite("3.png", new_tmap); +} + + + +/* + +int main(){ + + Mat img,tmap; + // my_vector_of_vectors_t samples, indm, Euu; + string img_path = "../../data/input_lowres/net.png"; + img = imread(img_path, CV_LOAD_IMAGE_COLOR); // Read the file + + string tmap_path = "../../data/trimap_lowres/Trimap1/net.png"; + tmap = imread(tmap_path, CV_LOAD_IMAGE_GRAYSCALE); + + Mat new_tmap = tmap.clone(); + trimming(tmap, tmap, new_tmap, tmap, true); +} + +*/ diff --git a/modules/alphamat/test/test_infoflow.cpp b/modules/alphamat/test/test_infoflow.cpp new file mode 100644 index 00000000000..701df8a58cb --- /dev/null +++ b/modules/alphamat/test/test_infoflow.cpp @@ -0,0 +1,131 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. +#include "test_precomp.hpp" +#include + + +namespace opencv_test { namespace { + +#define SAVE(x) imwrite(folder + "output.png", x); + +static const double numerical_precision = 0.05; // 95% of pixels should have exact values + +TEST(Alphamat_infoFlow, regression) +{ + string folder = string(cvtest::TS::ptr()->get_data_path()) + "alphamat/"; + string image_path = folder + "img/elephant.png"; + string trimap_path = folder + "trimap/elephant.png"; + string reference_path = folder + "reference/elephant.png"; + + Mat image = imread(original_path, IMREAD_COLOR); + Mat trimap = imread(original_path, IMREAD_COLOR); + Mat reference = imread(expected_path, IMREAD_GRAYSCALE); + + ASSERT_FALSE(image.empty()) << "Could not load input image " << original_path; + ASSERT_FALSE(trimap.empty()) << "Could not load input trimap " << trimap_path; + ASSERT_FALSE(reference.empty()) << "Could not load reference image " << reference_path; + + ASSERT_EQ(image.rows, trimap.rows) << "Height of image and trimap dont match"; + ASSERT_EQ(image.cols, trimap.cols) << "Height of image and trimap dont match"; + + Mat result; + infoFlow(image, trimap, result, true, true); + + SAVE(result); + + double errorINF = cvtest::norm(reference, result, NORM_INF); + EXPECT_LE(errorINF, 1); + double errorL1 = cvtest::norm(reference, result, NORM_L1); + EXPECT_LE(errorL1, reference.total() * numerical_precision) << "size=" << reference.size(); +} + +TEST(Alphamat_infoFlow, regression) +{ + string folder = string(cvtest::TS::ptr()->get_data_path()) + "alphamat/"; + string image_path = folder + "img/elephant.png"; + string trimap_path = folder + "trimap/elephant.png"; + string reference_path = folder + "reference/elephant.png"; + + Mat image = imread(original_path, IMREAD_COLOR); + Mat trimap = imread(original_path, IMREAD_COLOR); + Mat reference = imread(expected_path, IMREAD_GRAYSCALE); + + ASSERT_FALSE(image.empty()) << "Could not load input image " << original_path; + ASSERT_FALSE(trimap.empty()) << "Could not load input trimap " << trimap_path; + ASSERT_FALSE(reference.empty()) << "Could not load reference image " << reference_path; + + ASSERT_EQ(image.rows, trimap.rows) << "Height of image and trimap dont match"; + ASSERT_EQ(image.cols, trimap.cols) << "Height of image and trimap dont match"; + + Mat result; + infoFlow(original, result, true, false); + + SAVE(result); + + double errorINF = cvtest::norm(reference, result, NORM_INF); + EXPECT_LE(errorINF, 1); + double errorL1 = cvtest::norm(reference, result, NORM_L1); + EXPECT_LE(errorL1, reference.total() * numerical_precision) << "size=" << reference.size(); +} + + +TEST(Alphamat_infoFlow, regression) +{ + string folder = string(cvtest::TS::ptr()->get_data_path()) + "alphamat/"; + string image_path = folder + "img/elephant.png"; + string trimap_path = folder + "trimap/elephant.png"; + string reference_path = folder + "reference/elephant.png"; + + Mat image = imread(original_path, IMREAD_COLOR); + Mat trimap = imread(original_path, IMREAD_COLOR); + Mat reference = imread(expected_path, IMREAD_GRAYSCALE); + + ASSERT_FALSE(image.empty()) << "Could not load input image " << original_path; + ASSERT_FALSE(trimap.empty()) << "Could not load input trimap " << trimap_path; + ASSERT_FALSE(reference.empty()) << "Could not load reference image " << reference_path; + + ASSERT_EQ(image.rows, trimap.rows) << "Height of image and trimap dont match"; + ASSERT_EQ(image.cols, trimap.cols) << "Height of image and trimap dont match"; + + Mat result; + infoFlow(original, result, false, true); + + SAVE(result); + + double errorINF = cvtest::norm(reference, result, NORM_INF); + EXPECT_LE(errorINF, 1); + double errorL1 = cvtest::norm(reference, result, NORM_L1); + EXPECT_LE(errorL1, reference.total() * numerical_precision) << "size=" << reference.size(); +} + + +TEST(Alphamat_infoFlow, regression) +{ + string folder = string(cvtest::TS::ptr()->get_data_path()) + "alphamat/"; + string image_path = folder + "img/elephant.png"; + string trimap_path = folder + "trimap/elephant.png"; + string reference_path = folder + "reference/elephant.png"; + + Mat image = imread(original_path, IMREAD_COLOR); + Mat trimap = imread(original_path, IMREAD_COLOR); + Mat reference = imread(expected_path, IMREAD_GRAYSCALE); + + ASSERT_FALSE(image.empty()) << "Could not load input image " << original_path; + ASSERT_FALSE(trimap.empty()) << "Could not load input trimap " << trimap_path; + ASSERT_FALSE(reference.empty()) << "Could not load reference image " << reference_path; + + ASSERT_EQ(image.rows, trimap.rows) << "Height of image and trimap dont match"; + ASSERT_EQ(image.cols, trimap.cols) << "Height of image and trimap dont match"; + + Mat result; + infoFlow(original, result, false, false); + + SAVE(result); + + double errorINF = cvtest::norm(reference, result, NORM_INF); + EXPECT_LE(errorINF, 1); + double errorL1 = cvtest::norm(reference, result, NORM_L1); + EXPECT_LE(errorL1, reference.total() * numerical_precision) << "size=" << reference.size(); +} +}} //namespace diff --git a/modules/alphamat/test/test_main.cpp b/modules/alphamat/test/test_main.cpp new file mode 100644 index 00000000000..cda238cef3b --- /dev/null +++ b/modules/alphamat/test/test_main.cpp @@ -0,0 +1,11 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. +#include "test_precomp.hpp" +#include + +#if defined(HAVE_HPX) + #include +#endif + +CV_TEST_MAIN(".") \ No newline at end of file diff --git a/modules/alphamat/test/test_precomp.hpp b/modules/alphamat/test/test_precomp.hpp new file mode 100644 index 00000000000..1e95f4cb209 --- /dev/null +++ b/modules/alphamat/test/test_precomp.hpp @@ -0,0 +1,9 @@ +// This file is part of OpenCV project. +// It is subject to the license terms in the LICENSE file found in the top-level directory +// of this distribution and at http://opencv.org/license.html. +#ifndef __OPENCV_TEST_PRECOMP_HPP__ +#define __OPENCV_TEST_PRECOMP_HPP__ + +#include "opencv2/ts.hpp" + +#endif diff --git a/modules/alphamat/trimap/doll.png b/modules/alphamat/trimap/doll.png new file mode 100644 index 00000000000..36a76526c97 Binary files /dev/null and b/modules/alphamat/trimap/doll.png differ diff --git a/modules/alphamat/trimap/donkey.png b/modules/alphamat/trimap/donkey.png new file mode 100644 index 00000000000..d5dfbf13a98 Binary files /dev/null and b/modules/alphamat/trimap/donkey.png differ diff --git a/modules/alphamat/trimap/elephant.png b/modules/alphamat/trimap/elephant.png new file mode 100644 index 00000000000..b415c097f45 Binary files /dev/null and b/modules/alphamat/trimap/elephant.png differ diff --git a/modules/alphamat/trimap/net.png b/modules/alphamat/trimap/net.png new file mode 100644 index 00000000000..d64a9d42402 Binary files /dev/null and b/modules/alphamat/trimap/net.png differ diff --git a/modules/alphamat/trimap/pineapple.png b/modules/alphamat/trimap/pineapple.png new file mode 100644 index 00000000000..0733654d4d7 Binary files /dev/null and b/modules/alphamat/trimap/pineapple.png differ diff --git a/modules/alphamat/trimap/plant.png b/modules/alphamat/trimap/plant.png new file mode 100644 index 00000000000..89678946fe2 Binary files /dev/null and b/modules/alphamat/trimap/plant.png differ diff --git a/modules/alphamat/trimap/plasticbag.png b/modules/alphamat/trimap/plasticbag.png new file mode 100644 index 00000000000..630f603b665 Binary files /dev/null and b/modules/alphamat/trimap/plasticbag.png differ diff --git a/modules/alphamat/trimap/troll.png b/modules/alphamat/trimap/troll.png new file mode 100644 index 00000000000..3c8aa85690a Binary files /dev/null and b/modules/alphamat/trimap/troll.png differ