LandmarkPredictionAlgorithm Class Reference

#include <ImFusion/ML/LandmarkPredictionAlgorithm.h>

Algorithm to predict landmarks in an image. More...

Inheritance diagram for LandmarkPredictionAlgorithm:

Detailed Description

Algorithm to predict landmarks in an image.

This algorithm is based on two independent components: (1) the ML model used for producing a blob representing the probability distribution of the landmarks location (one channel per type). (2) the post-processing method to extract a landmark point from the blobs. All the components are configurable from the controller. Optionally you can omit to predict the blobs and directly use the input image to extract landmarks, provided the input image can be interpreted as a blob heatmap. Names assigned to the landmarks can be assigned from the model configuration: the field LabelNames represents a vector of strings, each one will be assigned to one landmark type (i.e. one channel in the network prediction).

Public Types
enum class	ModelType { HEATMAP_REGR = 0 , VOTING_SCHEME = 1 }
	Different definitions of the extracted landmark in case the model type is based on a voting scheme.
Public Types inherited from Algorithm
enum	Status {   Unknown = -1 , Success = 0 , Error = 1 , InvalidInput ,   IncompleteInput , OutOfMemoryHost , OutOfMemoryGPU , UnsupportedGPU ,   UnknownAction , AbortedByUser , User = 1000 }
	Status codes. More...

Public Member Functions
	LandmarkPredictionAlgorithm (const SharedImageSet *image=nullptr)
	Constructor.
	~LandmarkPredictionAlgorithm () override
	Destructor.
void	setImage (const SharedImageSet *image)
	image setter
void	compute () override
	Execute the algorithm.
void	configure (const Properties *p) override
	Set one or multiple properties.
void	configuration (Properties *p) const override
	Retrieve the properties of this object.
OwningDataList	takeOutput () override
	Generated output is added to the data list.
std::unique_ptr< SharedImageSet >	runModel (const SharedImageSet *img)
	Run the model and retrieve the label names.
std::unique_ptr< SharedImageSet >	aggregateVotesIntoBlobs (const SharedImage &voteImage) const
	Create blobs by aggregating the voting vectors Aggregate the polls returned by the voting scheme model and create one blob per channel.
std::vector< std::vector< vec3 > >	extractLandmarksFromBlobs (const SharedImage &blobs) const
	Given the predicted blobs, aggregates them into a collection of points.
const auto &	landmarks () const
	Get the detected landmarks.
auto &	getKeypointExtractor ()
const auto &	getKeypointExtractor () const
Public Member Functions inherited from Algorithm
	Algorithm ()
	Default constructor will registers a single "compute" action that calls compute() and returns status().
virtual void	setProgress (Progress *progress)
	Sets a Progress interface the algorithm can use to notify observers about its computing progress.
Progress *	progress () const
	Returns the progress interface if set.
virtual int	status () const
	Indicates the status of the last call to compute().
virtual bool	survivesDataDeletion (const Data *) const
	Indicates whether the algorithm can handle (partial) deletion of the specified data, by default this checks whether the data is in the input list.
const FactoryInfo &	factoryInfo () const
	Returns the record describing how this Algorithm was instantiated by the AlgorithmFactory.
void	setFactoryInfo (const FactoryInfo &value)
	Sets the record describing how this Algorithm was instantiated by the AlgorithmFactory.
Status	runAction (const std::string &id)
	Run the action with name id if it exists.
const std::vector< Action > &	actions ()
	Get a mapping from Action id to Action as registered in this algorithm.
Public Member Functions inherited from Configurable
virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.
void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().
void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.
	Configurable (const Configurable &rhs)
	Configurable (Configurable &&rhs) noexcept
Configurable &	operator= (const Configurable &)
Configurable &	operator= (Configurable &&) noexcept

Static Public Member Functions
static bool	createCompatible (const DataList &data, Algorithm **a=nullptr)
	If data is compatible with algorithm return true. If a is not zero, create algorithm with input data.
Static Public Member Functions inherited from Algorithm
static bool	createCompatible (const DataList &data, Algorithm **a=nullptr)
	Factory function to check algorithm compatibility with input data and optionally instantiate it.

Public Attributes
Parameter< bool >	p_predictBlobs = {"predictBlobs", true, this}
	Parameters.
Parameter< std::string >	p_configFile = {"predictionConfigFile", "", this}
	model config file
Parameter< bool >	p_showBlobs = {"showRegressedBlobs", false, this}
	whether to show the predicted landmark blobs
Parameter< ModelType >	p_modelType = {"modelType", ModelType::HEATMAP_REGR, this}
	type of ML model to be run
Public Attributes inherited from Algorithm
Signal	signalOutputChanged
	Signal should be emitted by Algorithms when their output/result has changed.
Signal	signalParametersChanged
	Signal should be emitted by Algorithms when their parameter configuration has changed.
Public Attributes inherited from Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' signalValueChanged signal was emitted.

Protected Attributes
const SharedImageSet *	m_image
std::vector< std::vector< std::unique_ptr< PointCloud > > >	m_predictedLandmarks
std::unique_ptr< SharedImageSet >	m_regressedBlobs
std::unique_ptr< KeypointExtractionAlgorithm >	m_keypointExtractor
	Keypoint extractor for processing the regressed blobs.
std::vector< std::string >	m_landmarkNames
Protected Attributes inherited from Algorithm
std::string	m_name
	Algorithm name.
Progress *	m_progress = nullptr
	Non-owing pointer to a progress interface. May be a nullptr.
FactoryInfo	m_factoryInfo = {}
	Record describing how this algorithm was instantiated by the AlgorithmFactory.
int	m_status = Status::Unknown
	Algorithm status after last call to compute()
std::vector< Action >	m_actions
	Map of key given by the id of the action, of the available actions of this algorithm.
Protected Attributes inherited from Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Additional Inherited Members
Protected Member Functions inherited from Algorithm
void	loadDefaults ()
void	registerAction (const std::string &id, const std::string &guiName, const std::function< Algorithm::Status(void)> &action)
	Register an action to be run via runAction.
template<typename D>
void	registerAction (const std::string &id, const std::string &guiName, Algorithm::Status(D::*action)(void))
	Template version of runAction that can be used with a pointer to a member function.
void	registerAction (const Action &action)
	Register an action.

Member Function Documentation

compute()

void compute ( )

overridevirtual

Execute the algorithm.

Implements Algorithm.

configure()

void configure ( const Properties * p )

overridevirtual

Set one or multiple properties.

Reimplemented from Configurable.

configuration()

void configuration ( Properties * p ) const

overridevirtual

Retrieve the properties of this object.

Reimplemented from Configurable.

takeOutput()

OwningDataList takeOutput ( )

overridevirtual

Generated output is added to the data list.

Reimplemented from Algorithm.

aggregateVotesIntoBlobs()

std::unique_ptr< SharedImageSet > aggregateVotesIntoBlobs

(

const SharedImage &

voteImage

)

const

Create blobs by aggregating the voting vectors Aggregate the polls returned by the voting scheme model and create one blob per channel.

This function is to be used only for the results of the Voting Scheme model.

extractLandmarksFromBlobs()

std::vector< std::vector< vec3 > > extractLandmarksFromBlobs

(

const SharedImage &

blobs

)

const

Given the predicted blobs, aggregates them into a collection of points.

Specifically, the outer vector lists the different landmark classes (number of blob channels), while the inner vector lists the diffent instances of the same landmark class found in a single blob channel.

Member Data Documentation

p_predictBlobs

Parameter<bool> p_predictBlobs = {"predictBlobs", true, this}

Parameters.

whether to run a ML model on the input image, or use the input directly as blobs for keypoint extraction

---

The documentation for this class was generated from the following file:

• ImFusion/ML/LandmarkPredictionAlgorithm.h