Landmark Prediction Algorithm

Predict the position of one or several landmarks in an image.

Input

The algorithm requires a single image as input (e.g. a CT scan) on which a machine learning model should predict the landmarks’ position.

Alternatively, it is also possible to skip the model prediction and directly provide the heatmap as input (in which case Predict Blobs should be set to False). In this case, the input must contain float values and a number of channels depending on the specified Model Type.

Output

The algorithm creates a point cloud annotation for every type of landmark detected; each point cloud may contain one or several detected landmarks. The output blobs of the model may also be added to the data widget if the Show Regressed Blobs flag is set.

Description

This algorithm is based on two independent components: the first is the ML model used for producing a blob representing the probability distribution of the landmark location, the second is the post-processing method to extract a landmark point from the blobs. Optionally you can omit to predict the blobs and directly use the input image to extract landmarks, provided the input image can be interpreted as a relevant heatmap or vector field.

The following parameters may be used to modify the default behavior of the algorithm:

  • Predict Blobs indicates whether a model must be run on the input image. This must be true if the input is an original image (e.g. a CT scan), and false if the input is already the output of a prediction (e.g. a heatmap).

  • Prediction Config File defines the path to the model YAML file (only active if Predict Blobs is true). More information on the YAML format can be found in our tutorial here.

  • Model Type controls how the algorithm should interpret the model prediction and can have the following values:

    • Heatmap Regression assumes that the prediction is a heatmap with float values: local maxima with high values correspond to pixel with high likelihood to correspond to a landmark. Different landmark types may be encoded in different channels of the heatmap.
    • Voting Scheme assumes that the prediction is a 2 or 3-channel vector field with float values: each pixel votes for the pixel with a relative position corresponding to the vector field. A heatmap is then estimated by accumulating the votes of all pixels (see Heatmap Regression).

  • Show Regressed Blobs controls whether the prediction map should be added to the data model (for visualization purposes).