#include <ImFusion/ML/Operations/SplitIntoPatchesOperation.h>

Operation which splits the input image into overlapping patches for sliding window inference. More...

Inheritance diagram for SplitIntoPatchesOperation:

Detailed Description

Operation which splits the input image into overlapping patches for sliding window inference.

The step size is used to compute valid patch positions that cover the full input image. The sampling behavior is controlled by the patch_step_size parameter, which is used to compute the patch offsets in the input image as a fraction of the specified patch size.

patch_size: Target size of the patches to be extracted as [Width, Height, Slices]. patch_step_size [0, 1]: Controls the step size between patches as a fraction of the patch size. In cases where the input image is a multiple of the patch size, a step size of 1.0 means no overlapping patches, while a lower step size means a higher number of overlapping patches. Example: If the input image is 100x100 and the patch size is 50x50, a patch_step_size of 1.0 will result in a 2x2 grid of non-overlapping patches, while a patch_step_size of 0.5 will result in a 3x3 grid of patches, with a 25 pixel overlap between adjacent patches. In cases where the input image is not a multiple of the ROI size, a step size of 1.0 indicates the optimal way of splitting the image in the least possible number of patches. Example: If the input image is 100x100 and the ROI size is 40x40, a patch_step_size of 1.0 will result in a 3x3 grid of patches, with a 10 pixel overlap between adjacent patches. Conversely, a patch_step_size of 0.5 will result in a 4x4 grid of patches, with a 20 pixel overlap between adjacent patches. padding_mode: Specifies the padding mode used when the input image is smaller than the specified patch size. In this case, the image is padded to the patch size with the specified padding mode.

Note: This operation uses the SplitROISampler internally.

Public Member Functions
	SplitIntoPatchesOperation (vec3i patchSize=vec3i::Ones(), double patchStepSize=0.8, PaddingMode paddingMode=PaddingMode::Mirror)

std::shared_ptr< SharedImageSet >	processImages (std::shared_ptr< SharedImageSet > input) const override
	Splits input image into patches.

DataItem	sampleFromROI (const DataItem &item, const RegionOfInterest &roi) const
	Splits input item into patches.

bool	doesNotModifyInput () const override
	Returns whether the operation is guaranteed to not modify its input element, either because it computes its output out-of-place or because it is a no-op.

bool	configure (const Properties &properties) override
	Configure an Operation with the given properties and return whether the configuration was successful.

virtual void	process (DataItem &item)
	Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden)

virtual std::shared_ptr< SharedImageSet >	process (std::shared_ptr< SharedImageSet > input)
	Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden)

virtual std::shared_ptr< KeypointSet >	process (std::shared_ptr< KeypointSet > input)
	Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden)

virtual std::shared_ptr< BoundingBoxSet >	process (std::shared_ptr< BoundingBoxSet > input)
	Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden)

virtual std::unique_ptr< SharedImageSet >	process (std::unique_ptr< SharedImageSet > input) final
	Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden)

Public Member Functions inherited from Operation
	Operation (std::string name, ProcessingPolicy processingPolicy)
	Derived classes must specify a name and whether, by default, the operation should also be applied to label maps.

void	addTemporaryPreProcessHook (PreProcessHook hook)
	Add a temporary hook that will only be active for the next process call and then automatically removed after the DataItem is processed.

void	addTemporaryPostProcessHook (PostProcessHook hook)
	Add a temporary hook that will only be active for the next process call and then automatically removed after the DataItem is processed.

virtual bool	checkRequiredItemsTypes (const DataItem &item) const
	Checks if the data item holds fields with types required by the operation.

const std::string &	name () const
	Returns the name of the operation.

virtual Properties	configuration () const
	Return the Operation configuration.

void	setActiveFields (std::optional< std::unordered_set< std::string > > activeFields)

std::optional< std::unordered_set< std::string > >	activeFields () const

virtual void	setProcessingPolicy (ProcessingPolicy policy)

ProcessingPolicy	processingPolicy () const

std::optional< uint32_t >	seed () const
	Return the current seed (if there is one)

virtual void	seedRandomEngine (uint32_t seed)
	Set a seed for the random generator.

ML::ComputingDevice	computingDevice () const
	Get the computing device.

virtual void	setComputingDevice (ML::ComputingDevice device)
	Set the computing device selection strategy.

void	configFailed (const std::string &missingParam) const
	Helper function to show an error message due to a bad configuration.

void	logDeprecatedParam (const std::string &oldName, const std::string &newName) const
	Helper function to print a warning because of a deprecated parameter has been specified.

bool	errorOnUnexpectedBehaviour () const
	Get the policy whether to treat unexpected behavior warnings as errors.

virtual void	setErrorOnUnexpectedBehaviour (bool error)
	Treat unexpected behavior warnings as errors.

void	registerParameter (ParameterBase *param)
	Register a parameter so that the operation knows about it, and can automatically configure it (unless it has been marked as manually configured parameter via the Operation::setManuallyConfiguredParameters method).

std::vector< ParameterBase * >	parameters () const
	Return the list of registered parameters.

virtual bool	supportsInversion () const
	Whether the operation supports inversion. Returns False unless the operation derives from `InvertibleOperation`.

const std::string &	recordIdentifier () const
	Get the operation's record identifier.

virtual void	setRecordIdentifier (const std::string &recordIdentifier)
	Set the operation's record identifier.

const std::string &	logDomain () const
	Log domain for AdvancedParameter.

std::optional< std::unordered_set< std::string > >	selectedFields (const DataItem &item)
	Get the fields on which the operation will be applied.

Public Attributes
OpParam< vec3i >	p_patchSize = {"patch_size", vec3i::Ones(), this, ParamRequired::Yes}
	Size of the patches to be extracted as [Width, Height, Slices].

OpParam< double >	p_patchStepSize = {"patch_step_size", 0.8, this, ParamRequired::No}
	Controls the step size between patches as a fraction of the patch size. Using nnUnet default value.

OpParam< PaddingMode >	p_paddingMode = {"padding_mode", PaddingMode::Mirror, this, ParamRequired::No}
	This might be used if the image to split is actually smaller than the specified patch size.

Additional Inherited Members
Public Types inherited from Operation
enum	ProcessingPolicy { EverythingExceptLabels = 0 , Everything , OnlyLabels }
	Policy used by default when selecting on the fields on which the operation will be applied Note that "Labels" here refers to the target tag, not to be confused with Data::Modality::LABEL. More...

using	PreProcessHook = std::function<void(Operation&, DataElement*)>

using	PostProcessHook = std::function<void(Operation&, DataElement*)>

Static Public Member Functions inherited from Operation
static std::string	processingPolicyToString (const ProcessingPolicy policy)

static ProcessingPolicy	stringToProcessingPolicy (const std::string &s)

static std::unique_ptr< Operation >	createFromFactories (const Operation::Specs &specs)
	Helper function to create an operation from any factory (both C++ and Python factories are tested)

Protected Member Functions inherited from Operation
virtual std::shared_ptr< BoundingBoxSet >	processBoxes (std::shared_ptr< BoundingBoxSet > input) const
	Virtual function that every subclass should override.

virtual std::shared_ptr< KeypointSet >	processPoints (std::shared_ptr< KeypointSet > input) const
	Virtual function that every subclass should override.

virtual std::shared_ptr< SharedImageSet >	processVectors (std::shared_ptr< SharedImageSet > input) const
	Virtual function that every subclass should override.

virtual std::shared_ptr< TensorSet >	processTensors (std::shared_ptr< TensorSet > input) const

void	throwOperationError (const std::string &msg) const
	Helper function to throw an exception due to a runtime error.

void	warnOperationUnexpectedBehaviour (const std::string &msg) const
	Helper function to warn about a behavior different than asked.

bool	inputIsEmptyOrNull (const SharedImageSet *input) const
	Helper function to check if the input is empty or null and print a warning if so.

virtual bool	useGPU (const SharedImageSet *input) const
	Helper function that returns which device the operation should choose based on its configuration and a given input.

void	prepareInputForDevice (SharedImageSet &input) const
	Make sure the input is on the correct device for the current device strategy.

virtual bool	allowChannelBatchOnGPU () const
	Whether the operation implements channel-batching so that it can still be run on input with more than 4 channels.

bool	configureOnly (const Properties &properties, const std::vector< ParameterBase * > &paramSelection) noexcept
	Helper method to configure only a subset of parameters.

void	setManuallyConfiguredParameters (const std::vector< ParameterBase * > &manuallyConfiguredParams)
	Disable the auto-configuration of some of the parameters so that they can be parsed manually (in the configure function of the derived class).

void	applyPreProcessHooks (DataElement *element)
	Apply the current pre-process hooks to an element.

void	applyPostProcessHooks (DataElement *element)
	Apply the current post-process hooks to an element.

Protected Attributes inherited from Operation
std::string	m_name

ProcessingPolicy	m_processingFieldsPolicy
	Flag specifying whether labels should be processed by default (when no active field has been specified)

std::optional< std::unordered_set< std::string > >	m_activeFields
	Run the Operation only on those fields (if empty, will select suitable fields based on the current processing policy) when calling process()

std::unordered_set< std::string >	m_alreadyWarned
	Set of warnings about unexpected behavior that have already been printed.

bool	m_errorOnUnexpectedBehaviour = false
	Whether to throw an exception instead of warning about unexpected behavior.

std::optional< uint32_t >	m_seed

Random::Generator	m_randGenerator

ML::ComputingDevice	m_device = ML::ComputingDevice::GPUIfOpenGl

std::vector< ParameterBase * >	m_params
	All registered parameters.

std::vector< ParameterBase * >	m_manuallyConfiguredParams
	All registered parameters that should not be configured automatically.

std::unordered_map< std::string, ElementType >	m_requiredFieldsTypes
	List of all required field types used by checkRequiredItemsTypes.

std::string	m_recordIdentifier
	User-provided unique identifier for this operation, used for recording processing history and inversion.

std::vector< PreProcessHook >	m_preProcessHooks

std::vector< PostProcessHook >	m_postProcessHooks

std::vector< PreProcessHook >	m_temporaryPreProcessHooks
	Pre-process hooks that will be used for the next processing call only.

std::vector< PostProcessHook >	m_temporaryPostProcessHooks
	Post-process hooks that will be used for the next processing call only.

Member Function Documentation

◆ processImages()

std::shared_ptr< SharedImageSet > processImages ( std::shared_ptr< SharedImageSet > input ) const

overridevirtual

Splits input image into patches.

Reimplemented from Operation.

◆ sampleFromROI()

DataItem sampleFromROI	(	const DataItem &	item,
		const RegionOfInterest &	roi ) const

Splits input item into patches.

For each ImageElement in the item, the function processImages is applied to the element. / All other element types are replicated to match the batch size of the splitted image elements. void process(DataItem& item) override;

/ Compute all the regions of interest for splitting the input image into patches. / This function is useful when the user wants to split the image into patches in a lazy fashion, / and then manually specify the ROI to use for sampling.

See also: sampleFromROI std::vector<RegionOfInterest> computeROIs(const SharedImageSet* image) const;

/ Samples a patch from the input item using a single ROI. / This function allows to split the image elements in the input item in a lazy fashion. / The user can precompute the ROIs using computeROIs, and then lazily produce patches by manually specifying the ROI to use for sampling. All non-image element types are cloned.

◆ doesNotModifyInput()

bool doesNotModifyInput ( ) const

inlineoverridevirtual

Returns whether the operation is guaranteed to not modify its input element, either because it computes its output out-of-place or because it is a no-op.

This can be used to decide whether a copy of the input should be made before calling the Operation::process function. Note that this only relates to the overloads taking specific elements as inputs (e.g. images, keypoints, etc.), since the Operation::process(DataItem& item) function always modifies its input.

Reimplemented from Operation.

◆ configure()

bool configure ( const Properties & properties )

overridevirtual

Configure an Operation with the given properties and return whether the configuration was successful.

Reimplemented from Operation.