Multi Modal Object Tracking
===========================

Tracks rigid object motion through a sequence of registered RGB-D frames using sparse features, optical flow, and 3D correspondence refinement.

Input
-----

Two or three image sets (same frame count and image size):

* **Color** — RGB ``UByte`` images, at least two frames; frame 0 is the reference.
* **Depth** — single-channel ``Float`` depth maps aligned with the color frames.
* **Mask** (optional) — single-channel label image; when provided, only features on the labeled object (default label 1) are used.

Each color and depth set needs a :ref:`camera calibration data component<cameracalibrationdatacomponent>` (or equivalent RGB-D calibration) so depth can be lifted to 3D.

Output
------

Registered point clouds and object poses relative to the reference frame. The controller can show live color, depth, and tracking views while processing runs in the background.

Description
-----------

For each frame after the reference, the algorithm detects features (optionally restricted by the mask), matches them to the previous keyframe window, lifts matches to 3D using depth, and estimates a rigid transform with RANSAC Kabsch refinement. Optical flow propagates features between keyframes. A reference mesh can be built from the tracked object for visualization.

Press **Compute** to start sequential processing; press again to stop an active run.

Tracking parameters
-------------------

**Early stopping match count percentage threshold** — fraction of reference keypoints matched before the keyframe search stops early.

**Minimum match count** — minimum number of accepted feature matches required to track a frame.

**Max keyframe count** — maximum keyframes kept in the sliding window.

**Optical flow uncertainty threshold** — discard optical-flow correspondences above this uncertainty.

**RANSAC Threshold** — inlier distance threshold (millimeters) for the Kabsch RANSAC loop.

**Max RANSAC iterations** — upper bound on RANSAC iterations per frame.

**Crop point clouds to label** — when a mask is used, limit output point clouds to labeled pixels.

Detector, matcher, and pruner settings
--------------------------------------

The properties panel exposes three nested groups from the :ref:`Feature Detection` pipeline. They run on each keyframe comparison **before** matches are lifted to 3D and passed to the RANSAC step above.

RIDE with Mask Detector Parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Learning-based RIDE keypoints and descriptors (see the ``RIDE`` entry under :ref:`Feature Detection`). When a mask input is provided, detection is limited to pixels with **Mask label** (default ``1``).

* **Max features** — upper bound on keypoints per frame (tracking default ``4000``).
* **Rotation invariance** — rotate descriptors to a canonical orientation (tracking default: off).
* **Dense non maximum suppression** — suppress nearby keypoints on a dense grid (tracking default: on).
* **Dense non maximum suppression radius** — NMS radius in pixels (tracking default ``3``).

Matcher Parameters
^^^^^^^^^^^^^^^^^^

Brute-force descriptor matching between the current frame and a keyframe in the sliding window. See the ``Brute Force`` matcher section in :ref:`Feature Detection`.

* **Norm** — distance measure for descriptor comparison (tracking default: ``DotProduct``, suited to RIDE).
* **Cross check** — keep a match only if each keypoint is the other's best match (tracking default: on).
* **Ratio threshold** — Lowe ratio test; matches above this ratio are rejected.

Pruner Parameters
^^^^^^^^^^^^^^^^^

Grid-based motion statistics (GMS) pruning to remove matches inconsistent with a smooth 2D motion model. See the ``Grid-based Motion Statistics`` section in :ref:`Feature Detection`.

* **Consider rotation** — allow rotational motion when scoring match consistency.
* **Consider scale** — allow scale change in the motion model.
* **Threshold factor** — multiplied with the median motion statistic to set the inlier cutoff; higher values prune more aggressively (tracking default ``3``, lower than the standalone Feature Detection default).