Feature Detection

Runs feature detection and matching on the input images

Input

One 2D image set. Only 8-bit grayscale and RGB images are supported.

Output

Visualization of detected features and feature correspondences.

Description

The algorithm implements a feature detection and matching pipeline. In a first step features are detected in the input images. In a second step these features can be sampled. Then they are matched using a feature matcher. The matches can be pruned using different constraints.

The detectors detect feature points and compute a feature descriptor.

  • SIFT
    • Max features. Maximum number of features to detect in image.
    • Octave layers. The number of layers in each octave.
    • Contrast threshold. The threshold value to filter out features coming from low-contrast regions. The higher the value, the less number of features are produced.
    • Edge threshold. The threshold value to filter out edge-like features. The higher the value, the more number of features are produced.
    • Sigma. Sigma of the Gaussian applied to the first image octave.
  • ORB
    • Edge threshold. The border size after which the features are not detected.
    • FAST threshold. Threshold value of the FAST descriptor detection.
    • First level. The level of the source of the source image in the pyramid.
    • Max features. Maximum number of features to detect in image
    • N levels. Number of levels in the image pyramid.
    • Patch size. Size of the patch used for the BRIEF descriptors.
    • Scale factor. Pyramid decimation ratio. If the value is 2, the next level image in the pyramid is twice smaller.
    • Score type. The algorithm used for ranking the descriptiveness. ORB uses Harris score by default.
    • WTA K. The number of points used to produce each element of the oriented BRIEF descriptor.
  • ShiTomasi
    • Max features. Maximum number of features to detect in image.
    • Block size. Size of an average block for computing a derivative covariation matrix over each pixel neighborhood.
    • Gradient size. Size of an average block for computing gradient over each pixel neighborhood.
    • Quality level. Parameter characterizing the minimal accepted quality of image corners. The parameter value is multiplied by the best corner quality measure, which is the minimal eigenvalue or the Harris function response. The corners with the quality measure less than the product are rejected.
    • Min distance. Minimum possible Euclidean distance between the returned corners.
    • Harris detector free param. Free parameter of the Harris detector.
    • Use Harris detector. Whether to use Shi-Tomasi or Harris Corner.
  • RIDE
    • Max features. Maximum number of features to detect in image.
    • Rotation invariance. Whether to invariantize to rotations. If checked, the algorithm will compute the orientation of each feature and rotate the feature descriptor accordingly.
    • Dense non maximum suppression. Whether to use dense non maximum suppression (NMS). This is different to the standard NMS, as it applies NMS on the dense grid of features, rather than on the sparse set of detected features.
    • Dense non maximum suppression radius. The radius of the dense NMS.

Some of the detectors support an adaptive thresholding approach to try to fill the image grid with at least one feature per cell. This works as follows:

  1. The detector runs the detection function of feature detector with no limit on the number of features and sorts them according to some criteria.
  2. The detector checks which grid cells have at least 1 feature in them, and if not all of the cells are filled, the algorithm decreases the threshold.
  3. Steps 1-2 are repeated either until all cells are filled or until the threshold reaches its minimum.
  4. Final features will contain first Max features best keypoints + a best feature per cell.

Available parameters are:

  • Iteratively adapt threshold. Indicates whether the adaptive thresholding should be applied.
  • Grid cell size. The cell size of the image grid. The algorithm will attempt to fill each cell with at least one feature.
  • Minimum threshold. The lowest value of the threshold to be tested.
  • Threshold step. The threshold step used to decrease the threshold from iteration to iteration.

The feature sampler filters the features by some criteria. Currently supported sampler is Non Maximum Suppression. Features are first sorted by their response. Then for every feature point it is checked whether there are other features within the specified Radius. And if so, these features are removed.

The matcher establishes correspondences between features. Currently, there are two matchers available:

  • Brute Force. Uses nearest neighbourhood algorithm.

    • Norm. Selects the norm used for distance computation. Use L1 or L2 for SIFT and Hamming for ORB.
    • Cross check. If checked, instead of searching for the k nearest neighbours, it searches for the nearest neighbour for each keypoints bidirectionally. The match is only retained only if the nearest neighbours agree.
  • Grid based. Searches for the best match within the radius.

    • Norm. Selects the norm used for distance computation.
    • Window size. Radius of the search area.
    • Check orientation. If checked, keypoint orientation is taken into account during matching.

Match features. If checked, matches the features between the images whose indices are set in Images to match.

Match sequentially. If checked (also Match features is needed to be checked), matches the descriptors in input image set in a sequential manner.

Images to match. Pair of image indices to match in the input image set.

The pruning model removes correspondences which do not fulfill the pruning constraints.

  • None. Don’t apply any pruning.

  • Homography. The underlying assumption of this model is that all points lie on the same scene plane. Inlier threshold specifies how far in pixels a correspondences allowed to be from the position predicted by the model to be counted as an inlier.

  • Fundamental Matrix. The underlying model is a general two view constraint. The scene must not only consist of a plane. In this case use the Homography model. Inlier threshold specifies how far in pixels a correspondences allowed to be from the position predicted by the model to be counted as an inlier.

  • Auto. Automatically selects between Homography and Fundamental Matrix depending on the score computed for each model, and prunes with a corresponding selected pruner.

  • Match Score. Prunes matches either by selecting N best matches or by removing the matches that have a score (distance) higher than the threshold, or both. Use pruning based on max number of matches enables the first option, while Use pruning based on score threshold enables the second. Max number of matches specifies the number of best matches to be kept when using the first option. Match score threshold specifies the maximum match score for the second option.

  • Grid-based Motion Statistics. Applies motion smoothness as a pruning criteria. Recommended to be used with lots of matches (~10k). Based on: /// Bian, JiaWang, Wen-Yan Lin, Yasuyuki Matsushita, Sai-Kit Yeung, Tan-Dat Nguyen, and Ming-Ming Cheng. “Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4181-4190. 2017.

    • Consider rotation If checked, the algorithm will consider rotation as a possible motion. Otherwise, only translation (or also scaling) is considered.
    • Consider scale If checked, the algorithm will consider scale as a possible motion. Otherwise, only translation (or also rotation) is considered.
    • Threshold factor The threshold factor is multiplied by the median of the motion statistics to obtain the threshold. The higher the value, the more matches are pruned.

When the Compute button is pressed with Match features enabled, a user can view each match and its score interactively by checking Show matches interactively. A user can then scroll through matches by changing the value in Current match - this will highlight a corresponding match in the image pair. Show all matches toggles the visualization of all matches in the image pair. The button Print match statistics logs the range, mean, median and standard deviation of match scores (distances).