Texturing

This algorithm creates a texture or vertex colors for a given mesh using color images, typically from the RGB-D sequence the mesh was originally created from.

Input

A mesh and corresponding set of RGB-D images. The images need to be 8bit RGB and have an elementwise RGB-D data component.

Note that in order to select both meshes and images for an algorithm, you have to attach the mesh to the image (using “move to” in the meshes context menu), then again use the context menu of the mesh to launch the algorithm.

Output

A copy of the input mesh with a texture or vertex colors created from the input image.

Description

The algorithm creates a texture or vertex colors for the mesh from the given images using the specified method.

The following methods and parameters are available:

• MVS:

  Unless the input images already have an ExplicitMask attached, the algorithm creates a mask by rendering the mesh into the image plane as depth map, then using sudden jumps in the depth values to estimate object boundaries. The parts of the input image covered by the mask are then used to create the texture for the mesh using the MVS libary.

  • Erosion radius: The mask indicating which pixels belong to an object is twice morphologically expanded with this radius, in pixels.

  • Depth discontinuity threshold: How much a pixel in the depth map must differ from any of its neighbours to be classified as object boundary, in Millimeters.

  • Depth visibility threshold: Determines the optimizer step length if Optimize poses is enabled, in Millimeters for the position and degrees for the rotation.

  • Data term: Affects how the texturing is optimized internally by the MVS library.

  • Outlier removal: Outlier removal strategy used internally by MVS.

  • Tone mapping: Tone mapping appraoch used internally by MVS.

  • Geometric visibility test: If enabled, MVS does a clipping test with the mesh geometry to determine which parts are visible. Otherwise only camera and basic scene geometry is used to determine visibility.

  • Global seam leveling: If enabled, MVS runs the seam leveling procedure described in the paper “Seamless Mosaicing of Image-Based Texture Maps” by Ivanov and Lempitsky

  • Local seam leveling: If enabled, MVS does a local seam leveling, i.e. tries to match colors at the edges where information from different input images connects.

  • Hole filling: If enabled, MVS tries to fill smaller holes in the mesh while texturing.

  • Keep unseen faces: If this is enabled MVS creates a separate texture patch for faces not appearing in any of the others.

  • Inpainting color: The default color for the texture patch created by Keep unseen faces.

  • Optimize poses: If enabled, the pose of the mesh relative to the camera may be varied to obtain a better match with the images.

  • Show keyframe assignment: If enabled, the red channel of the face colors of the output mesh contains the index of the associated node in the texturing graph.

  • Min. cluster size: All clusters in the texturing graph with less than this number of elements will be merged into neighbouring clusters.

• ColorMapOptimization:

  This mode uses open3d to perform both the masking of the input images and the actual vertex coloring.

  • Erosion radius: Specifies the half-kernel size for the dilation applied on the visibility mask image.

  • Depth discontinuity threshold: It’s often desirable to ignore points where there is an abrupt change in depth value. First the depth gradient image is computed, points are considered to be invisible if the depth gradient magnitude is larger than this value.

  • Depth visibility threshold: When the difference of a point’s depth value in the RGB-D image and the point’s depth value in the 3D mesh is greater than this value, the point is marked as invisible to the camera producing the RGB-D image.

  • Subdivision resolution: If this value is greater than zero, the mesh is subdivided (upsampled) with a maximum edge length in Millimeters of this value before texturing.

  • Max. iterations: Number of iterations when optimizing the extrinsic camera parameters (and optionally distortion if Non-rigid optimization is enabled) during texturing.

  • Non-rigid optimization: When set, open3d also optimizes an image wrapping field when assigning color values, i.e. allows for more complex distortion of the images.

  • Max. depth: Points with depth in Millimeters larger than this value in a RGB-D will be marked as invisible for the camera producing that RGB-D image. Select a proper value to include necessary points while ignoring unwanted points such as the background.

• Blending:

  This approach averages the color of a vertex of all input images it appears in, weighted by how well the vertex normal aligns with the camera view direction.

  • Erosion radius: The mask indicating which pixels belong to an object is twice morphologically expanded with this radius, in pixels.

  • Depth discontinuity threshold: How much a pixel in the depth map must differ from any of its neighbours to be classified as object boundary, in Millimeters.

  • Depth visibility threshold: If the difference between vertex position and the depth map at that position in Millimeters exceeds this value, the image will be skipped when computing the average color for that vertex.

  • Subdivision resolution: If this value is greater than zero, the mesh is subdivided (upsampled) with a maximum edge length in Millimeters of this value before texturing.