#include <ImFusion/ML/Engine.h>

Generic interface for machine learning models serialized by specific frameworks (PyTorch, ONNX, etc.). More...

Inheritance diagram for ImFusion::ML::Engine:

Detailed Description

Generic interface for machine learning models serialized by specific frameworks (PyTorch, ONNX, etc.).

This class is used by the MachineLearningModel to forward the prediction request to the framework that was used to serialize the model.

C++ implementations of this interface are provided through dedicated plugins that encapsulate the framework-specific logic and libraries required for model deserialization and execution. See for instance the TorchPlugin and OnnxRuntimePlugin.

Python bindings are also available, enabling engine implementation directly in Python. This approach is useful for prototyping and Python SDK users. See imfusion.machinelearning.engines for examples of Python engine implementations.

Public Member Functions
	Engine (const std::string &name)
virtual DataItem	predict (const DataItem &input)=0
	Main method to compute predictions of all classes on a DataItem containing heterogeneous elements.
std::string	name () const
	Returns the name of the operation as defined in the factories.
virtual void	configure (const Properties *properties) override
	Configure the Engine according to the properties.
virtual void	configuration (Properties *properties) const override
	Serialize the current object configuration into the given Properties object.
virtual bool	isIdentical (const Engine *other) const
	Check whether this engine instance is the same as other.
virtual void	checkInputFields (const DataItem &input)
	Check that inputs specified in the config are present in the input item.
virtual void	checkOutputFields (const DataItem &input)
	Check that outputs specified in the config are present in the predicted item.
virtual std::vector< ExecutionProvider >	availableProviders () const =0
	Returns which provider is supported by the engine and available on the host.
virtual std::optional< ExecutionProvider >	provider () const =0
	Returns which provider is used by the engine to run the model, empty if none available.
std::optional< ByteBuffer >	loadModelArtifact () const
	Convenience function to load the model artifact specified in p_modelFile either from the model file or from a resource repository.
Public Member Functions inherited from ImFusion::Configurable
virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.
void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().
void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.
	Configurable (const Configurable &rhs)
	Configurable (Configurable &&rhs) noexcept
Configurable &	operator= (const Configurable &)
Configurable &	operator= (Configurable &&) noexcept

Static Public Member Functions
static std::vector< std::string >	availableEngines (EngineLanguage language=EngineLanguage::Any)
	List all registered inference engines for a given implementation language.
static bool	isRegistered (const std::string &engineName, bool quiet=true, EngineLanguage language=EngineLanguage::Any)
	Check if an engine is registered for the selected implementation language.
static std::unique_ptr< Engine >	create (const std::string &engineName, const Properties &properties, EngineLanguage language=EngineLanguage::Any)
	Factory function to create an engine for the selected implementation language.

Public Attributes
Parameter< std::string >	p_modelFile = {"ModelFile", "", nullptr}
	Parameters shared by all engines.
Parameter< bool >	p_forceCPU = {"ForceCPU", false, nullptr}
	Whether to force run on CPU even if GPU is present.
Parameter< std::vector< std::string > >	p_inputFields = {"EngineInputFields", std::vector<std::string>{}, nullptr}
	Input field names expected by the model run by the engine.
Parameter< std::vector< std::string > >	p_outputFields = {"EngineOutputFields", std::vector<std::string>{}, nullptr}
	Output field names returned by the model run by the engine.
Parameter< std::vector< std::string > >	p_ignoreOutputFields = {"EngineOutputFieldsToIgnore", std::vector<std::string>{}, nullptr}
	Output field names to be ignored by the engine.
Parameter< std::string >	p_version = {"Version", "", nullptr}
	Version of the engine used to train and save the model.
Parameter< bool >	p_logInputToDataLogger = {"LogInputToDataLogger", false, this}
	When true, each engine input `SharedImageSet` is sent to the Data Logger.
Public Attributes inherited from ImFusion::Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' signalValueChanged signal was emitted.

Protected Member Functions
virtual void	init (const Properties &properties)
	Configures the engine from properties + connect signals.
virtual void	onModelFileChanged ()
	Function that gets called when p_modelFile changes.
virtual void	onForceCPUChanged ()
	Function that gets called when p_forceCPU changes.
void	logInputToDataLogger (const DataItem &input) const
	Sends all `SISBasedElement` inputs to the Data Logger when `p_logInputToDataLogger` is enabled.
virtual void	connectSignals ()
	Connect the signals like onModelFileChanged, onForceCPUChanged, ... (We do not do this automatically because children class need to call this after they register their parameters).

Protected Attributes
EngineConfiguration	m_config
	Configuration object.
std::string	m_name = ""
	Name of the engine interface (e.g. "torch").
Protected Attributes inherited from ImFusion::Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Member Function Documentation

◆ predict()

virtual DataItem ImFusion::ML::Engine::predict ( const DataItem & input )

pure virtual

Main method to compute predictions of all classes on a DataItem containing heterogeneous elements.

Implemented in ImFusion::OnnxRuntime::OnnxEngine, and ImFusion::TorchEngine.

◆ configure()

virtual void ImFusion::ML::Engine::configure ( const Properties * properties )

overridevirtual

Configure the Engine according to the properties.

If the engine cannot be configured, a runtime_error should be raised

Reimplemented from ImFusion::Configurable.

◆ configuration()

virtual void ImFusion::ML::Engine::configuration ( Properties * p ) const

overridevirtual

Serialize the current object configuration into the given Properties object.

The default implementation will do so automatically for all registered Parameter and SubProperty instances.

See also: configure() for the inverse functionality

Reimplemented from ImFusion::Configurable.

◆ availableProviders()

virtual std::vector< ExecutionProvider > ImFusion::ML::Engine::availableProviders ( ) const

pure virtual

Returns which provider is supported by the engine and available on the host.

Implemented in ImFusion::OnnxRuntime::OnnxEngine, and ImFusion::TorchEngine.

◆ provider()

virtual std::optional< ExecutionProvider > ImFusion::ML::Engine::provider ( ) const

pure virtual

Returns which provider is used by the engine to run the model, empty if none available.

Implemented in ImFusion::OnnxRuntime::OnnxEngine, and ImFusion::TorchEngine.

◆ availableEngines()

std::vector< std::string > ImFusion::ML::Engine::availableEngines ( EngineLanguage language = EngineLanguage::Any )

static

List all registered inference engines for a given implementation language.

Parameters

language Cpp, Python, or Any for the union of both.

◆ isRegistered()

bool ImFusion::ML::Engine::isRegistered	(	const std::string &	engineName,
		bool	quiet = true,
		EngineLanguage	language = EngineLanguage::Any )

static

Check if an engine is registered for the selected implementation language.

When the silent flag is set to false, in case no engine is found the function will print a detailed error message.

◆ create()

std::unique_ptr< Engine > ImFusion::ML::Engine::create	(	const std::string &	engineName,
		const Properties &	properties,
		EngineLanguage	language = EngineLanguage::Any )

static

Factory function to create an engine for the selected implementation language.

With EngineLanguage::Any, all registered engine will be considered.

Note: For the torch and onnx engines, the C++ implementation have precendence over the corresponding python one, in case both are available. If you want to only allow the Python implementation, set the env variable IMFUSION_PLUGIN_BLACKLIST=Torch;OnnxRuntime, to avoid the C++ engine to be registered.

◆ loadModelArtifact()

std::optional< ByteBuffer > ImFusion::ML::Engine::loadModelArtifact ( ) const

Convenience function to load the model artifact specified in p_modelFile either from the model file or from a resource repository.

Returns: The model artifact as a ByteBuffer, or std::nullopt if the model artifact could not be loaded

◆ init()

virtual void ImFusion::ML::Engine::init ( const Properties & properties )

protectedvirtual

Configures the engine from properties + connect signals.

This function must be called in the constructor of all implementations of this interface.

◆ onModelFileChanged()

virtual void ImFusion::ML::Engine::onModelFileChanged ( )

inlineprotectedvirtual

Function that gets called when p_modelFile changes.

Reimplemented in ImFusion::OnnxRuntime::OnnxEngine, and ImFusion::TorchEngine.

◆ onForceCPUChanged()

virtual void ImFusion::ML::Engine::onForceCPUChanged ( )

inlineprotectedvirtual

Function that gets called when p_forceCPU changes.

Reimplemented in ImFusion::OnnxRuntime::OnnxEngine, and ImFusion::TorchEngine.

Member Data Documentation

◆ p_modelFile

Parameter<std::string> ImFusion::ML::Engine::p_modelFile = {"ModelFile", "", nullptr}

Parameters shared by all engines.

Model file, serialized by the specific framework

The documentation for this class was generated from the following file:

ImFusion/ML/Engine.h

Detailed Description

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Member Function Documentation

◆ predict()

◆ configure()

◆ configuration()

◆ availableProviders()

◆ provider()

◆ availableEngines()

◆ isRegistered()

◆ create()

◆ loadModelArtifact()

◆ init()

◆ onModelFileChanged()

◆ onForceCPUChanged()

Member Data Documentation

◆ p_modelFile