 |
ImFusion C++ SDK 4.5.0
|
|
ImFusion C++ SDK 4.5.0
|
Plugin for computer vision functionality. More...
Plugin for computer vision functionality.
The ImFusionVision plugin offers a comprehensive set of 2D and 3D computer vision algorithms including feature and marker detection, single and stereo camera calibration, optical flow and stereo computation and much more. It combines traditional techniques with modern algorithms powered by state-of-the-art machine learning models.
Topics | |
| ICP and Mesh Alignment | |
| Comprehensive guide to point cloud and mesh alignment using ICP and MeshAlignmentAlgorithm. | |
| Marker Detection | |
| Comprehensive guide to marker detection. | |
| Feature Detection | |
| Comprehensive guide to Feature Detection and Matching. | |
| Monocular Depth Estimation | |
| Guide to monocular depth estimation from single RGB images. | |
| Multi-View Reconstruction (MapAnything) | |
| Guide to 3D reconstruction from unposed RGB images using the MapAnything model. | |
| Optical Flow | |
| Guide to optical flow estimation from RGB image sequences. | |
| Stereo Reconstruction | |
| Guide to 3D reconstruction from a sequence of stereo images using the StereoReconstructionAlgorithm class. | |
| Camera Calibration | |
| Comprehensive guide to Camera Calibration. | |
| Stereo Structure | |
| Comprehensive guide to Stereo Image. | |
Classes | |
| class | ImFusion::CameraCalibrationDataComponent |
| A data component storing mainly the intrinsic calibration of a pinhole camera Information such as image size, extrinsic poses and distortions are also stored Debug values like MRE and standard deviations estimated for intrinsic parameters, which are used for evaluating the calibration results during the calibration process are also saved here. More... | |
| class | ImFusion::ColorMapOptimization |
| Provides color map optimization for textured mesh generation from RGB data. More... | |
| class | ImFusion::CreateStereoSharedImageSetAlgorithm |
| The algorithm creates a stereo shared image set from two shared image sets The algorithm take over the ownership of the input two image sets It is recommended to first run this algorithm to convert the two image sets into a stereo image set before using any stereo-related algorithm. More... | |
| class | ImFusion::DepthColorWarping |
| DepthColorWarping provides functionality for warping between depth and color images using camera calibration and extrinsic parameters. More... | |
| class | ImFusion::DepthMapToPointCloudAlgorithm |
| DepthMapToPointCloudAlgorithm converts depth maps (optionally with color images) into point clouds. More... | |
| class | ImFusion::HandEyeCalibrationAlgorithm |
| Performs hand-eye calibration between two tracking sequences. More... | |
| class | ImFusion::ImageSharpnessEstimationAlgorithm |
| Algorithm for estimating sharpness of the image (based on paper "The Blur Effect: Perception and Estimation with a New No-Reference Perceptual Blur Metric"). More... | |
| class | ImFusion::InterlacedToStereoAlgorithm |
| This algorithm converts a interlaced image into two shared image sets or a stereo image set Input image should have even number of lines. More... | |
| class | ImFusion::LeastSquaresPoseGraphSolution |
| Pose graph optimization with Least Squares solution based on g2o's implementation The first pose is the reference pose and will stay fixed, all other poses are free to change. More... | |
| class | ImFusion::MotionAveragingPoseGraphSolution |
| Pose graph optimization with motion averaging, this is to be utilized in MAICP The first pose is the reference pose and will stay fixed, all other poses are free to change. More... | |
| class | ImFusion::MultiCameraCalibrationDataComponent |
| A data component storing the matrix representing a projective transformation to a reference coordinate system for multi-sensor-calibration. More... | |
| class | ImFusion::MVSTexturing |
| Provides multi-view stereo (MVS) texturing for 3D meshes using RGB-D image sets. More... | |
| class | ImFusion::PointCloudPlaneFittingAlgorithm |
| Provides plane fitting and distance computation for point clouds. More... | |
| class | ImFusion::PointCloudToDepthMapAlgorithm |
| Provides conversion of a dense point cloud to a depth map. More... | |
| class | ImFusion::PoseGraphOptimization |
| Pose graph optimization The first pose is the reference pose and will stay fixed, all other poses are free to change. More... | |
| class | ImFusion::RGBDCalibrationDataComponent |
| Stores the rigid transform between depth and color cameras of an RGB-D pair. More... | |
| class | ImFusion::SideBySideToStereoAlgorithm |
| This Algorithm converts a side-by-side images set to a Stereo Image Set. More... | |
| class | ImFusion::SimulatorFrame |
| Container for a rendered simulator frame of the SimulatorRenderer. More... | |
| class | ImFusion::SimulatorRenderer |
| Renderer for simulating RGB-D sensor output from 3D models. More... | |
| class | ImFusion::StereoCalibrationDataComponent |
| A data component storing the registration matrix between the left and right camera of a stereo camera pair. More... | |
| class | ImFusion::StereoImage |
| This class stores a pair of images, taken by a stereo camera at the same time point. More... | |
| class | ImFusion::StereoImageExtractSelection |
| Algorithm to extract the current StereoSharedImageSet selection into a new object instance. More... | |
| class | ImFusion::StereoImageReduceToSelection |
| Algorithm for reducing a StereoSharedImageSet to its selected samples. More... | |
| class | ImFusion::StereoSharedImageSet |
| This class is the main high-level container for stereo image data set. More... | |
| class | ImFusion::TexturingAlgorithm |
| Algorithm for texturing 3D meshes using RGB-D image data. More... | |
| class | ImFusion::VisionDataAnnotationFactory |
| Factory providing data annotations, icons, attributes, and cloning for Vision data types. More... | |
| class | ImFusion::BlobDetector |
| Detects locally connected regions ("blobs") in grayscale or RGB images. More... | |
| class | ImFusion::CameraCalibrationAlgorithm |
| Camera calibration algorithm Description: More... | |
| class | ImFusion::CameraCalibrationSettings |
| The class stores all parameters/settings for camera calibration. More... | |
| class | ImFusion::CameraRegistrationAlgorithm |
| Camera registration algorithm Description: This algorithm performs camera registration for a stereo camera system using known fiducial markers (e.g., chessboard or a ChArUco board) to estimate the relative pose between the left and right cameras. More... | |
| class | ImFusion::DepthMapToMeshAlgorithm |
| DepthMapToMeshAlgorithm converts depth maps (optionally with color and mask images) into triangle meshes. More... | |
| class | ImFusion::ORBFeatureDetector |
| Implementation of the ORB feature detection and description algorithm. More... | |
| class | ImFusion::SIFTFeatureDetector |
| Implementation of the SIFT feature detection and description algorithm. More... | |
| class | ImFusion::ShiTomasiFeatureDetector |
| Implementation of the Shi-Tomasi corner detection algorithm. More... | |
| class | ImFusion::IdentitySampler |
| Identity sampler, returns input keypoints unchanged. More... | |
| class | ImFusion::NMSSampler |
| Non-Maximum Suppression sampler. More... | |
| class | ImFusion::BruteForceMatcher |
| Brute-force matcher for feature descriptors. More... | |
| class | ImFusion::GridBasedMatcher |
| Grid-based matcher for feature descriptors. More... | |
| class | ImFusion::IdentityMatchPruner |
| Identity match pruner, returns input matches unchanged. More... | |
| class | ImFusion::HomographyMatchPruner |
| Pruner using homography inlier threshold. More... | |
| class | ImFusion::FundamentalMatrixMatchPruner |
| Pruner using fundamental matrix inlier threshold. More... | |
| class | ImFusion::AutomaticMatchPruner |
| Automatic pruner selecting between homography and fundamental matrix based on decomposition score. More... | |
| class | ImFusion::MatchScorePruner |
| Pruner based on match score and/or maximum number of matches. More... | |
| class | ImFusion::GMSPruner |
| Pruner applying motion smoothness as a criterion (GMS). More... | |
| class | ImFusion::FeatureDetectionAlgorithm |
| Feature detection algorithm. More... | |
| class | ImFusion::FeatureDetectorFactory |
| Factory for registering and instantiating FeatureDetector implementations. More... | |
| class | ImFusion::FeatureSamplerFactory |
| Factory for registering and instantiating FeatureSampler implementations. More... | |
| class | ImFusion::FeatureMatcherFactory |
| Factory for registering and instantiating FeatureMatcher implementations. More... | |
| class | ImFusion::FeatureMatchPrunerFactory |
| Factory for registering and instantiating FeatureMatchPruner implementations. More... | |
| class | ImFusion::HomographyAndFundamentalMatrixDecomposition |
| This class, given point correspondences from two images, estimates the homography and the fundamental matrix, then, depending on the score, decomposes one of them into rotation and translation. More... | |
| class | ImFusion::ImageUndistortion |
| This class enables the undistortion of images, i.e. More... | |
| class | ImFusion::ImageUndistortionAlgorithm |
| Algorithm for undistoring images, intrinsics and distortion parameters are retrieved from CameraCalibrationDataComponent, if not then user defined ones are used. More... | |
| class | ImFusion::MarkerConfiguration |
| Describes the configuration of a marker calibration target. More... | |
| class | ImFusion::MarkerConfigurationWidget |
| Widget for editing MarkerConfiguration used in marker detection and calibration workflows. More... | |
| class | ImFusion::MarkerDetection |
| Generic marker detection class for a wide range of marker types, such as Aruco, Charuco, Chessboard, Circle Board, AprilTag and STag. More... | |
| class | ImFusion::MarkerDetectionAlgorithm |
| Marker detection algorithm. More... | |
| class | ImFusion::MarkerGenerationAlgorithm |
| Algorithm for creating calibration markers. More... | |
| class | ImFusion::MarkerHandEyeCalibrationAlgorithm |
| Performs hand eye calibration of a sensor that provides tracking information (called "tracker") to a camera that captured images of a marker board (called "camera"). More... | |
| class | ImFusion::OpticalFlow |
| Base class for optical flow algorithms producing flow maps (2-channel floating point images). More... | |
| class | ImFusion::OpticalFlowFactory |
| Factory for registering and instantiating OpticalFlow backends. More... | |
| class | ImFusion::Vision::OpticalFlowAlgorithm |
| class | ImFusion::StereoReconstruction |
| Base class for stereo reconstruction algorithms producing disparity maps. More... | |
| class | ImFusion::StereoReconstructionFactory |
| Factory for registering and instantiating StereoReconstruction backends. More... | |
| class | ImFusion::StereoBlockMatchingReconstruction |
| Implementation of block matching stereo reconstruction It depends on block matching from OpenCV. More... | |
| class | ImFusion::StereoSemiGlobalBlockMatchingReconstruction |
| Implementation of Semi Global block matching stereo reconstruction It depends on Semi Global block matching from OpenCV. More... | |
| class | ImFusion::StereoReconstructionAlgorithm |
| This algorithm performs 3D reconstruction from a pair of stereo images. More... | |
| class | ImFusion::StereoRectification |
| Performs stereo rectification und unrectification on images. More... | |
| class | ImFusion::StereoRectificationAlgorithm |
| Stereo Rectification Algorithm. More... | |
| class | ImFusion::GlobalICP |
| Global ICP point cloud alignment. More... | |
| class | ImFusion::ICP |
| ICP-based point cloud alignment. More... | |
| class | ImFusion::PointCloudFilteringAlgorithm |
| Implements a flexible and extensible algorithm for filtering and processing point clouds. More... | |
| class | ImFusion::PointCloudOverlapAlgorithm |
| Computes the geometric overlap between multiple dense, calibrated point clouds. More... | |
| class | ImFusion::PoseGraphOptimizationAlgorithm |
| Pose graph optimization using motion-averaging or graph-based least squares solver. More... | |
| class | ImFusion::MeshAlignmentAlgorithm |
| Algorithm for aligning meshes and point clouds. More... | |
| class | ImFusion::MonocularDepthEstimation |
| Base class for monocular depth estimation algorithms producing depth maps and (optionally) also point clouds. More... | |
| class | ImFusion::MonocularDepthEstimationFactory |
| Factory for registering and instantiating MonocularDepthEstimation backends. More... | |
| class | ImFusion::MonocularDepthEstimationAlgorithm |
| Algorithm for estimating relative depth from RGB images. More... | |
| class | ImFusion::MonocularDepthEstimationDepthAnything |
| Monocular relative depth estimation using the Depth Anything V2 Small neural network. More... | |
| class | ImFusion::MonocularDepthEstimationCustomModel |
| Performs depth estimation on a monocular image using a traced machine learning model provided by the user. More... | |
| class | ImFusion::MultiViewReconstructionMapAnything |
| 3D reconstruction using the MapAnything TorchScript model. More... | |
| class | ImFusion::MultiViewReconstructionMapAnythingAlgorithm |
| Algorithm for 3D reconstruction using the MapAnything model. More... | |
| class | ImFusion::Vision::MLHelpers::VisionMLModelsFactory |
| Registry and loader for machine learning models used by Vision algorithms. More... | |
| class | ImFusion::MonocularStreamDepthEstimationAlgorithm |
| Performs real-time monocular depth estimation on a live image stream. More... | |
| class | ImFusion::MonocularStreamDepthEstimationController |
| GUI controller for MonocularStreamDepthEstimationAlgorithm. More... | |
| class | ImFusion::OpticalFlowRAFT |
| Dense optical flow backend based on RAFT (Recurrent All-Pairs Field Transforms). More... | |
| class | ImFusion::OpticalFlowSEARAFT |
| Dense optical flow backend based on SEARAFT, a simpler and faster RAFT variant (https://arxiv.org/abs/2405.14793). More... | |
| class | ImFusion::OpticalFlowCustomModel |
| Performs optical flow estimation using a traced machine learning model provided by the user. More... | |
| class | ImFusion::RIDEFeatureDetector |
| Implementation of "RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy" published at ICRA 2024: https://ieeexplore.ieee.org/abstract/document/10611381. More... | |
| class | ImFusion::StereoCGIReconstruction |
| Stereo disparity backend using the CGI-Stereo neural network. More... | |
| class | ImFusion::StereoReconstructionRAFT |
| Stereo disparity backend based on RAFT-Stereo (https://arxiv.org/abs/2109.07547). More... | |
| class | ImFusion::StereoReconstructionCustomModel |
| Performs stereo reconstruction using a traced machine learning model provided by the user. More... | |
| class | ImFusion::LiveHandEyeCalibrationAlgorithm |
| Takes two tracking streams, and allows to sample frames and to compute the calibration from them The expected input is a stream containing the transforms from base to hand (i.e. More... | |
| class | ImFusion::LiveHandEyeEvaluationAlgorithm |
| Takes two tracking streams and the respective calibration, and computes the chain error. More... | |
| class | ImFusion::CameraCalibrationSetterAlgorithm |
| The algorithm loads an existing calibration onto a given image set or image stream. More... | |
| class | ImFusion::CreateGroupedStereoImageStreamAlgorithm |
| The algorithm creates a GroupedStereoImageStream from two image streams and takes over their control Before using any stereo-related algorithms (e.g., depth estimation, disparity calculation, or 3D reconstruction), it is recommended to first run this algorithm to convert the two image streams into a stereo image stream. More... | |
| class | ImFusion::GroupedStereoImageStream |
| GroupedStereoImageStream simultaneously owns and controls its stereo sub-streams. More... | |
| class | ImFusion::PlaybackStereoImageStream |
| StereoImage-based stream created from the playback of a stereo image set. More... | |
| class | ImFusion::PlaybackStereoImageStreamAlgorithm |
| Creates a PlaybackStereoImageStream from a StereoSharedImageSet. More... | |
| class | ImFusion::SideBySideToStereoStreamAlgorithm |
| This class convert a stream with side-by-side image into a stereo image stream. More... | |
| class | ImFusion::StereoCalibrationSetterAlgorithm |
| The algorithm loads an existing calibration onto a given stereo image set or stereo image stream. More... | |
| class | ImFusion::StereoImageStream |
| Base class for streams based on stereo images. More... | |
| class | ImFusion::StereoImageStreamData |
| Stereo Image data streamed from a stereo camera. More... | |
| class | ImFusion::InterlacedToStereoStreamAlgorithm |
| This class convert a stream with interlaced image into a stereo image stream Input image should have even number of lines. More... | |
| class | ImFusion::LiveStereoCalibrationAlgorithm |
| Takes two image streams, and allows to sample frames and to compute the calibration from them Both streams have to see the same calibration board The output is the intrinsic calibration for each camera, transformation between the cameras and the deformation fields for stereo rectification Another variant of the algorithm takes image samples as an input. More... | |
| class | ImFusion::LiveTrackingAlgorithm |
| Live tracking algorithm for image based pose detectors Use the Configurable interface for setting up the detector. More... | |
| class | ImFusion::StereoStreamReconstructionController |
| GUI controller for StereoStreamReconstructionAlgorithm. More... | |
| class | ImFusion::StereoStreamToVideoRecorder |
| Records stereo image streams to two video files The algorithm internally uses VideoFileRecorder to save videos. More... | |
| class | ImFusion::StereoStreamToVideoAlgorithm |
| Algorithm to record stereo image streams to video files The algorithm internally uses VideoFileRecorder to save videos. More... | |
| class | ImFusion::MeshCollisionDetectionController |
| Controller for assigning meshes to collision groups and running MeshCollisionDetectionAlgorithm. More... | |