Feature Detection

Comprehensive guide to Feature Detection and Matching.

Collaboration diagram for Feature Detection:

Comprehensive guide to Feature Detection and Matching.

This page provides detailed information and code examples for detecting and matching features in 2D images using the ImFusion SDK. The feature detection module supports a variety of detectors, samplers, matchers, and pruning strategies, enabling robust and flexible pipelines for computer vision applications such as registration, tracking, and 3D reconstruction.

Feature Detection Overview

Feature detection is a fundamental step in many computer vision tasks. It involves identifying distinctive points (keypoints) in images and describing them with feature descriptors. These features can then be matched across images to establish correspondences, which are essential for tasks like image registration, structure-from-motion, and visual odometry estimation.

The ImFusion SDK provides a modular and extensible framework for feature detection, sampling, matching, and pruning. The main components are:

• FeatureDetector: Detects keypoints and computes descriptors.
• FeatureSampler: Filters or subsamples detected features.
• FeatureMatcher: Establishes correspondences between features in different images.
• FeatureMatchPruner: Removes unreliable matches based on geometric or statistical criteria.

Feature Detection Workflow

1. Detection: Extract keypoints and descriptors from input images using a chosen detector (e.g., SIFT, ORB, Shi-Tomasi, RIDE).
2. Sampling: Optionally filter or subsample features (e.g., using Non-Maximum Suppression).
3. Matching: Match features between images using a matcher (e.g., Brute Force, Grid-based).
4. Pruning: Remove unreliable matches using geometric or statistical constraints (e.g., Homography, Fundamental Matrix, GMS).

Feature Detectors

The following detectors are available:

• SIFT: Scale-Invariant Feature Transform. Detects robust, scale- and rotation-invariant features. Implements Lowe, David G. "Distinctive image features from scale-invariant keypoints." International journal of computer vision 60, no. 2 (2004): 91-110.
• ORB: Oriented FAST and Rotated BRIEF. Efficient binary descriptor, suitable for real-time applications. Implements Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. "ORB: An efficient alternative to SIFT or SURF" International conference on computer vision (2011): 2564-2571.
• Shi-Tomasi: Corner detector. Based on Shi, J. & Tomasi, C. "Good features to track." In 1994 Proceedings of IEEE conference on computer vision and pattern recognition, pp. 593-600. IEEE, 1994.
• RIDE: RIDE: Self-supervised learning of rotation-equivariant keypoint detection and invariant description for endoscopy. Parameters include max features, rotation invariance, dense non-maximum suppression, and dense NMS radius. Implements Mert Asim Karaoglu, Viktoria Markova, Nassir Navab, Benjamin Busam, and Alexander Ladikos. "RIDE: Self-supervised learning of rotation-equivariant keypoint detection and invariant description for endoscopy." In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 10764-10771. IEEE, 2024.

Some detectors support adaptive thresholding to ensure a uniform spatial distribution of features. This is controlled by parameters such as grid cell size, minimum threshold, and threshold step.

Feature Sampling

Feature samplers filter or subsample detected keypoints. The main sampler is:

• Non Maximum Suppression (NMS): Retains only the strongest keypoints within a specified radius, reducing redundancy.

Feature Matching

Feature matchers establish correspondences between features in different images:

• Brute Force Matcher: Finds nearest neighbors using a specified norm (L1, L2, Hamming). Supports cross-checking and ratio thresholding.
• Grid-based Matcher: Searches for best matches within a spatial window, optionally considering orientation.

Feature Match Pruning

Pruning removes unreliable matches using geometric or statistical models:

• None: No pruning.
• Homography: Assumes all points lie on a plane. Prunes matches based on inlier threshold.
• Fundamental Matrix: General two-view constraint. Prunes matches based on inlier threshold.
• Auto: Automatically selects between Homography and Fundamental Matrix based on model scores.
• Match Score: Prunes by keeping N best matches or those below a score threshold.
• Grid-based Motion Statistics (GMS): Prunes based on motion smoothness, suitable for large numbers of matches.

Feature Detection API

The main classes are:

• FeatureDetector
• FeatureDetectorWithAdaptiveThreshold
• FeatureSampler
• FeatureMatcher
• FeatureMatchPruner

Example: Detecting and Matching Features

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/FeatureDetection.h>
 
using namespace ImFusion;
 
// Load or create input images
SharedImage img1;
SharedImage img2;
 
// Create a feature detector (e.g., ORB)
ORBFeatureDetector detector;
 
// Detect features
std::vector<Keypoint> keypoints1 = detector.detect(img1);
std::vector<Keypoint> keypoints2 = detector.detect(img2);
 
// Optionally sample features
NMSSampler sampler;
keypoints1 = sampler.sample(keypoints1);
keypoints2 = sampler.sample(keypoints2);
 
// Match features
BruteForceMatcher matcher;
matcher.p_norm = BruteForceMatcher::Norm::Hamming;
std::vector<FeatureMatch> matches = matcher.match(keypoints1, keypoints2);
 
// Prune matches (e.g., using GMS)
GMSPruner pruner;
std::vector<FeatureMatch> prunedMatches = pruner.prune(keypoints1, keypoints2, matches, {img1.width(), img1.height()}, {img2.width(), img2.height()});

Feature Detection Parameters

Each detector, matcher, and pruner exposes parameters for fine-tuning. These can be set directly via the corresponding Parameter objects (e.g., detector.p_maxFeatures = 500;).

Best Practices

• Choose the detector and matcher based on your application (e.g., SIFT for robustness, ORB for speed).
• Use adaptive thresholding and NMS to ensure good spatial coverage and reduce redundancy.
• Always prune matches using geometric constraints for robust results.
• Visualize detected features and matches to verify correctness.

Troubleshooting

• If too few features are detected, lower the detection threshold or increase max features.
• If matches are poor, try a different matcher or adjust the norm/threshold parameters.
• For planar scenes, use Homography pruning; for general scenes, use Fundamental Matrix. Auto selects the relevant option automatically.
• For large numbers of matches or improved robustness, use GMS pruning.

See also

FeatureDetector Class, FeatureMatcher Class, FeatureMatchPruner Class