Metric Class Reference

#include <ImFusion/ML/Metric.h>

The Metric interface is a key part of our pipeline for standardized evaluations of a machine learning pipeline, or an algorithm in general. More...

Inheritance diagram for Metric:

Detailed Description

The Metric interface is a key part of our pipeline for standardized evaluations of a machine learning pipeline, or an algorithm in general.

Each class implementing this interface will be in charge of computing one or several statistics based on a target and a prediction: for instance the Dice coefficient between a ground truth label map and the output of an algorithm. Metrics may take as input any number of elements supported by the ML::DataItem class.

A metric usually only needs to be instantiated once (with optionally a given set of parameters) and can then be computed independently for any number of input. As shown in the example below, the most generic way to run a computation is to call the compute method with a DataItem that includes the expected elements provided by the data scheme (see dataScheme function). Note, however, that the derived classes may also expose some more dedicated and tailored functions.

DiceMetric diceComputer;
diceComputer.p_ignoreBackground = true;
 
DataItem data;
data.add("target", targetImageElement);
data.add("prediction", predictionImageElement);
std::vector<Record> metrics = diceComputer.compute(data);
// The variable `metrics` now contains as many Records as the number of frames in the target/prediction elements. Each Record is a map containing the returned metrics:
// metric[0] == { "dice_Label1": 0.556, "dice_Label2": 0.995 }
// metric[1] == { "dice_Label1": 0.721, "dice_Label2": 0.984 }
// The entries are encoded as Eigen::MatrixXd for the sake of generalizability even though most of them will typically be scalars .

Disclaimer: this is still a work-in-progress concept subject to changes. In the near future, we plan to introduce:

• a mechanism to compute such metrics on a complete set of datas and generate some report;
• an Algorithm to compute such metrics from the ImFusionSuite;
• a factory with a registry so that new metrics can be added by SDK users.

Public Types
using	Field = typename DataItem::Field
using	DataScheme = std::map<Field, ElementType>
using	Record = std::map<std::string, Eigen::MatrixXd>

Public Member Functions
virtual DataScheme	dataScheme () const =0
	Returns the expected content of the input data.
virtual std::vector< Record >	compute (const DataItem &testItem) const =0
	Generic method to compute the metric (each child class may have a dedicated and easier-to-use function) Result type is a vector (one record per frame) of maps (one entry per sub-metric)
bool	isCompatibleWith (const DataItem &item, bool strictCheck=false) const
	Checks if the input data item is compatible with the template metric.
Public Member Functions inherited from Configurable
virtual void	configure (const Properties *p)
	Configure this object instance by de-serializing the given Properties.
virtual void	configuration (Properties *p) const
	Serialize the current object configuration into the given Properties object.
virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.
void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().
void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.
	Configurable (const Configurable &rhs)
	Configurable (Configurable &&rhs) noexcept
Configurable &	operator= (const Configurable &)
Configurable &	operator= (Configurable &&) noexcept

Protected Member Functions
void	assertValidDataScheme (const DataItem &testItem) const
	Asserts if the input data item is compatible with the template metric.
void	throwMetricError (std::string msg) const
	Helper function to throw a dedicated error with more info.

Additional Inherited Members
Public Attributes inherited from Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' signalValueChanged signal was emitted.
Protected Attributes inherited from Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Member Function Documentation

dataScheme()

virtual DataScheme dataScheme ( ) const

pure virtual

Returns the expected content of the input data.

Implemented in DiceMetric, ImagewiseClassificationMetrics, PixelwiseClassificationMetrics, and SurfaceDistancesMetric.

compute()

virtual std::vector< Record > compute ( const DataItem & testItem ) const

pure virtual

Generic method to compute the metric (each child class may have a dedicated and easier-to-use function) Result type is a vector (one record per frame) of maps (one entry per sub-metric)

Implemented in DiceMetric, ImagewiseClassificationMetrics, PixelwiseClassificationMetrics, and SurfaceDistancesMetric.

isCompatibleWith()

bool isCompatibleWith	(	const DataItem &	item,
		bool	strictCheck = false ) const

Checks if the input data item is compatible with the template metric.

The strictCheck flag ensures that only the required elements are present.

assertValidDataScheme()

void assertValidDataScheme ( const DataItem & testItem ) const

protected

Asserts if the input data item is compatible with the template metric.

This function throws an MetricException if this is not the case.

---

The documentation for this class was generated from the following file:

• ImFusion/ML/Metric.h