Machine Learning

The machine learning (ML) plugin contains all generic algorithms related to the execution of machine learning models (in particular neural networks) in the ImFusion software. It also adds extra functionalities related to pre- and post-processing of models input and output, data augmentation, image sampling, etc.

This plugin contains the main pieces of the machine learning framework but is not self-sufficient. In order to execute ML models, additional plugins that natively integrate external frameworks are required. This can be done either in C++ or in Python (provided you have the imfusion-sdk python package). In either case, the resulting plugin will be available to the ImFusionSuite/SDK. See the page Machine Learning Model for more details.

The following ML frameworks are currently supported for C++ development:

Framework	Windows	Linux	MacOS Arm
Torch	Yes: CPU, CUDA	Yes: CPU, CUDA	Yes: CPU, MPS
ONNX Runtime	Yes: CPU, CUDA, DirectML	Yes: CPU, CUDA	No
TensorRT	Yes	Yes	No

Note

Previous versions of the ImFusion Suite might include CaffePlugin, TensorFlowPlugin or TOMAATPlugin; they are no longer supported.

In Python, we currently offer implementations for the following frameworks as part of the imfusion-sdk package:

• PyTorch

• ONNX Runtime

• Core ML

• OpenVINO

Note

The imfusion-sdk does not ship the pytorch, onnxruntime, coreml and so on. In order to use the corresponding framework, you need to install it (i.e. with pip). In this sense, we do not specify which device is supported since this depends on the python package flavor which is installed (i.e. torch-gpu or onnxruntime-silicon and so on).

The main algorithm of the ML plugin is Run MachineLearning Model. The Operations Sequence algorithm is also useful to check the pre- and post-processing of a model. All other algorithms are considered experimental or internal.