Overview

The vision module provides methods used for acquiring, processing, analyzing and understanding digital images. This includes (but is not limited to): camera calibration, image and point cloud filtering, feature detection, mesh reconstruction, etc…

It is strongly recommended to refer to data structures used in the vision module, namely Mesh, PointCloud and SharedImageSet.

Camera Calibration

The algorithm performs camera calibration based on a set of 2D images. By estimating the camera intrinsic matrix, distortion coefficients and transformation vectors by solving the equations that map 3D object points to their corresponding 2D image points in the image plane.

>>> from imfusion import vision
>>> image_set, *_  = imfusion.load(...)  
>>> marker_config = vision.marker_configuration.ChessboardInfo()
>>> marker_config.grid_size = (9, 7)
>>> marker_config.cell_size = (30, 30)

>>> vision.calibrate_camera(image_set, marker_config)

If the function executes successfully, it will assign a valid CameraCalibrationDataComponent to the image_set. It can be extracted in our example by running the following line:

>>> calibration_data = image_set.components.camera_calibration

More information about the marker configuration can be found in the documentation of ImFusion Suite

Mesh Alignment (ICP)

The Mesh/PointCloud algorithm minimizes the difference between the source and the target by iteratively refining the transformation (rotation and translation) applied to the source points.

>>> mesh_source, *_ = imfusion.load(...)  
>>> mesh_target, *_ = imfusion.load(...)  
>>> meshes = [mesh_source, mesh_target]
>>> vision.align(meshes, use_gpu=False, icp_algorithm=vision.AlignmentIcpAlgorithm.FAST_GLOBAL_REGISTRATION_OPEN3D)
[(-1.0, 2.7128763582966438e-17)]

The function align() updates the transformation matrix of the source mesh if ICP is run successfully:

>>> registration_matrix = mesh_source.matrix_to_world()

Ball Pivoting Surface Reconstruction

The function run the ball pivoting surface reconstruction [1] to create a mesh out of a point cloud.

[1] - Bernardini Fausto, Joshua Mittleman, Holly Rushmeier, Cláudio Silva, and Gabriel Taubin. “The ball-pivoting algorithm for surface reconstruction.” IEEE transactions on visualization and computer graphics 5, no. 4 (1999): 349-359.

>>> pc, *_ = imfusion.load(...)  
>>> mesh = vision.ball_pivoting_surface_reconstruction(pc)
>>> imfusion.save(mesh, tmp_path / "mesh.ply")