Stereo Stream Reconstruction

Performs stereo reconstruction on live stereo camera streams and emits a continuous depth output stream.

Input

Either:

Output

The primary result is a depth stream: each frame bundles two images — a depth map and the matching left-rectified RGB color image — for live inspection and point-cloud preview.

Optional debug outputs (see below) add separate disparity and/or rectified stereo preview streams to the data model.

Description

The algorithm wraps the same stereo reconstruction pipeline as Stereo Reconstruction, applied frame-by-frame to a live stream. By default Stereo Block Matching, Stereo Semi Global Block Matching, and the learning-based Stereo CGI and Stereo RAFT backends are available. Choose the active backend with Stereo reconstruction (under the nested Stereo Reconstruction property group). Only parameters belonging to the selected backend are shown in the properties panel.

Common parameters (all backends)

Every backend receives rectified stereo pairs first. These settings always apply:

Rectification alpha - Parameter controlling the rectification with values between 0 (output image only contains valid pixels) and 1 (output image contains all pixels including mask pixel). A value of -1 applies an automated process to choosing the alpha value.

Stereo Block Matching and Stereo Semi Global Block Matching

The following parameters (and additional OpenCV tuning options in the properties panel) apply only when one of these two backends is selected:

Block size - Size of the patch in pixels used for matching left and right streams. Only odd values are supported. Even values will be mapped to the next larger odd value.

Min disparity - Minimum disparity value in pixels to start search.

Num disparities - Maximum number of pixels to slide the patch over.

Uniqueness ratio - Controls how unique the match between the left and the right image needs to be in order to be accepted.

Stereo RAFT

These parameters apply only when Stereo RAFT is selected (default settings usually work well):

Init from previous computation - Marking this will store the computed disparity internally and subsequent computation will initialize the disparity from the previously stored disparity. Setting to true can help in better quality results in case of temporally smooth videos.

Number of refinements - Setting to a value greater than zero will perform that many refinement steps. Usually zero is enough. If required a value in range [1, 3] can further improve the results.

Downsample to train resolution (if larger) - Downsample inputs to the network’s training resolution when they are larger than that resolution.

Use smaller network - Uses a smaller machine learning model which is faster (but less accurate) than the default model.

Stereo CGI

When Stereo CGI is selected, disparity is computed with the CGI-Stereo neural network. Tuning is limited to the nested ML model types settings (backend selection, autocast, and related model options). Block-matching parameters do not apply.

Processing

Reconstruction may lag behind the live frame rate. The following parameters control how frames are queued and processed:

Queue size - Number of frames stored while waiting for reconstruction. When the queue is full, new frames are dropped.

Force blocking - When enabled, each frame is processed on the input thread instead of a background queue (higher latency, simpler timing).

Live point-cloud preview

Show point cloud - Toggles a live 3D point-cloud view in the algorithm display. The preview uses the depth map and left-rectified RGB color from the primary depth stream; it does not add a separate output stream.

Debug streams

These options add extra streams to the workspace after the algorithm runs:

Show disparity stream - Adds a live disparity preview stream (separate from the primary depth stream).

Show rectified streams - Adds a stereo stream with live left and right rectified previews (separate from the left-rectified RGB image already included in the depth stream).