Mesh Alignment

This algorithm performs point cloud to point cloud, mesh to pointloud, pointcloud to mesh and mesh to mesh 3D rigid registration. The algorithm internally updates data matrices such that input point clouds/meshes align better in an error metric sense. The last selected mesh/point cloud is taken as a reference, and all the others are sequentially aligned to it (on example of 3 objects: 2nd to 3rd, 1st to 2nd).

Input

Multiple point clouds or meshes or combinations of both.

Output

None. The input point clouds/meshes will have an adjusted transformation such that the input data is registered.

Description

The algorithm performs a check on the input data, such that depending on the presence of normals, “dense calibration” (point cloud originating from depth image with intrinsics for transformation into depth) or vertex colors, different options are available. Also the avaiable options may vary depending on the availability of Open3D or PCL in the framework.

The algorithm leaves the last object (point cloud or mesh) untouched and modifies all other ones in terms of their transformation matrix.

Several registration modes are supported, varying from own implementations of the ICP and versions from PCL or Open3D:

  • Point-to-Plane ICP (projective). Performs a point to plane ICP alignment with a projective search for matches. Requires all point clouds to have a dense calibration (camera intrinsics and a map from pixel of the originating depth image to the point index), and target point clouds to have normals. If the input data satisfies the requirements, this mode should be preferred over the non-projective variant, since matches are found much faster in this mode.
  • Point-to-Plane ICP. Performs a point to plane ICP alignment with a nearest neighbor search for matches. Requires target point clouds/meshes to have normals.
  • Point-to-Point ICP. Performs a point to point ICP alignment.
  • Manual Correspondences. In this mode a user is expected to manually select corresponding points on the input data by double-clicking on them in the 3D view. At least 3 correspondences are required from each data. The algorithm then aligns the input data based on these correspondences. This mode can be especially useful when the initial alignment of the data is far off. It allows to perform a rough initial alignment, and a user can afterwards choose to execute one of the other modes for fine-tuning.
  • Generalized ICP. Performs a generalized ICP alignment described in the article by Alex Segal et al.
  • RANSAC Global Registration. While ICP alignment methods are known as local registration methods requiring an initial rough alignment of the input data, this is a global registration method that doesn’t have this prerequisite. Instead, it should be used as an initial rough alignment step and then the transformations should be refined using local registration methods. The RANSAC global registration method works as follows. First, the input data is downsampled and the geometric features are extracted (see the article by Radu Bogdan Rusu at al for more details). In each RANSAC iteration, 3 random points are picked from the source data. Their corresponding points in the target data are detected by querying the nearest neighbor in the geometric feature space. False matches are rejected via pruning step. The remaining matches are then used to compute a transformation.
  • Fast Global Registration. A global registration that is faster than RANSAC Global Registration. Based on the article by Qian-Yi Zhou et al.
  • Colored ICP. Performs an ICP alignment using both, geometry and texture information. The algorithm is described in the article by Jaesik Park et al. Requires the input data to have colors.

Different modes support different choice of parameters from the following set:

  • Max corresp. distance. Defines a distance threshold for the points to be matched.
  • Max corresp. angle. Defines a threshold for the angle between the point normals for the points to be matched.
  • Max iterations. Defines a maximum number of ICP iterations.
  • Abort parameter tolerance. Defines a threshold for the matrix update. If the update is too small, the iterative procedure stops.
  • Overlap ratio. If the overlap ratio r is >0 and <1, the Max corresp. distance will be re-computed by taking the distance of the r-th match from the list of sorted initial matches.
  • Voxel size. Defines a voxel size for downsampling the input data.
  • Reciprocal corresp. If true, the point pair p1i and p2j will be considered a match only if the best correspondence for p1i is p2j and the best correspondence for p2j is p1i. In not checked, only the first part of the condition suffices.
  • Use GPU option attempts to improve performance by moving some of the computation onto the GPU.

Some modes offer additional functionalities for RMS error computation and match visualization. Availability of these functionalities may depend on the selected mode, number of input data, use of GPU and presence of normals.

  • Compute RMS calculates an RMS distance between the matched points.
  • Compute WRMS calculates a weighted RMS distance between the matched points. If point clouds have weights assigned to each point, these weights will be considered when computing this error. Otherwise, this functionality is unavailable.
  • Show matches visualizes detected matches by drawing lines in the 3D view. In order to see these lines better, a user may use a data transformation manipulator in order to temporarily move the data apart. The “Esc” key can then be used to undo this transformation. Match density allows to adjust the number of displayed matches.

The Reset button may be used to reset the matrices of the input data to their initial state. The Compute button triggers the actual alignment.