 |
ImFusion SDK 4.3
|
|
ImFusion SDK 4.3
|
#include <ImFusion/ML/Operations/RecombinePatchesOperation.h>
Operation to recombine image patches back into a full image. More...
Inheritance diagram for RecombinePatchesOperation:Operation to recombine image patches back into a full image.
This operation is typically used in conjunction with SplitIntoPatchesOperation to reconstruct a full image from its patches after processing (e.g., after neural network inference).
The operation handles overlapping patches by averaging the overlapping regions. For each output pixel, the final value is computed as the weighted average of all patches that contain that pixel. The weighting mode is specified by the RecombineMode parameter.
The operation requires input images to have a PatchesFromImageDataComponent that stores the location of each patch in the original image. This component is automatically added by the SplitIntoPatchesOperation or by the SplitROISampler.
Two recombination modes are supported:
Both GPU and CPU computing devices are supported, via the ComputingDevice parameter in Operation.
Public Member Functions | |
| RecombinePatchesOperation (RecombineMode mode=RecombineMode::Default, ComputingDevice device=ComputingDevice::GPUIfOpenGl) | |
| std::shared_ptr< SharedImageSet > | processImages (std::shared_ptr< SharedImageSet > input) const override |
| Virtual function that every subclass should override. | |
| bool | doesNotModifyInput () const override |
| Returns whether the operation is guaranteed to not modify its input element, either because it computes its output out-of-place or because it is a no-op. | |
| Public Member Functions inherited from Operation | |
| Operation (std::string name, ProcessingPolicy processingPolicy) | |
| Derived classes must specify a name and whether, by default, the operation should also be applied to label maps. | |
| virtual void | process (DataItem &item) |
| Main function that processes a DataItem in-place, may throw OperationException The data to be processed is determined by the active fields (if set) and the processing policy otherwise (if set), otherwise all fields are processed. | |
| virtual std::shared_ptr< SharedImageSet > | process (std::shared_ptr< SharedImageSet > input) |
| Utility function to directly apply an operation on a SharedImageSet irrespective of active fields and processing policy. | |
| virtual std::shared_ptr< KeypointSet > | process (std::shared_ptr< KeypointSet > input) |
| Utility function to directly apply an operation on a set of KeypointSet irrespective of active fields and processing policy. | |
| virtual std::shared_ptr< BoundingBoxSet > | process (std::shared_ptr< BoundingBoxSet > input) |
| Utility function to directly apply an operation on a set of BoundingBoxSet irrespective of active fields and processing policy. | |
| virtual std::unique_ptr< SharedImageSet > | process (std::unique_ptr< SharedImageSet > input) final |
| Utility function with unique_ptr as input and output to avoid breaking changes (must not be overridden) | |
| void | addTemporaryPreProcessHook (PreProcessHook hook) |
| Add a temporary hook that will only be active for the next process call and then automatically removed after the DataItem is processed. | |
| void | addTemporaryPostProcessHook (PostProcessHook hook) |
| Add a temporary hook that will only be active for the next process call and then automatically removed after the DataItem is processed. | |
| virtual bool | checkRequiredItemsTypes (const DataItem &item) const |
| Checks if the data item holds fields with types required by the operation. | |
| const std::string & | name () const |
| Returns the name of the operation. | |
| virtual bool | configure (const Properties &properties) |
| Configure an Operation with the given properties and return whether the configuration was successful. | |
| virtual Properties | configuration () const |
| Return the Operation configuration. | |
| void | setActiveFields (std::optional< std::unordered_set< std::string > > activeFields) |
| std::optional< std::unordered_set< std::string > > | activeFields () const |
| virtual void | setProcessingPolicy (ProcessingPolicy policy) |
| ProcessingPolicy | processingPolicy () const |
| std::optional< uint32_t > | seed () const |
| Return the current seed (if there is one) | |
| virtual void | seedRandomEngine (uint32_t seed) |
| Set a seed for the random generator. | |
| ML::ComputingDevice | computingDevice () const |
| Get the computing device. | |
| virtual void | setComputingDevice (ML::ComputingDevice device) |
| Set the computing device selection strategy. | |
| void | configFailed (const std::string &missingParam) const |
| Helper function to show an error message due to a bad configuration. | |
| void | logDeprecatedParam (const std::string &oldName, const std::string &newName) const |
| Helper function to print a warning because of a deprecated parameter has been specified. | |
| bool | errorOnUnexpectedBehaviour () const |
| Get the policy whether to treat unexpected behavior warnings as errors. | |
| virtual void | setErrorOnUnexpectedBehaviour (bool error) |
| Treat unexpected behavior warnings as errors. | |
| void | registerParameter (ParameterBase *param) |
| Register a parameter so that the operation knows about it, and can automatically configure it (unless it has been marked as manually configured parameter via the Operation::setManuallyConfiguredParameters method). | |
| std::vector< ParameterBase * > | parameters () const |
| Return the list of registered parameters. | |
| virtual bool | supportsInversion () const |
Whether the operation supports inversion. Returns False unless the operation derives from InvertibleOperation. | |
| const std::string & | recordIdentifier () const |
| Get the operation's record identifier. | |
| virtual void | setRecordIdentifier (const std::string &recordIdentifier) |
| Set the operation's record identifier. | |
| const std::string & | logDomain () const |
| Log domain for AdvancedParameter. | |
| std::optional< std::unordered_set< std::string > > | selectedFields (const DataItem &item) |
| Get the fields on which the operation will be applied. | |
Public Attributes | |
| OpParam< RecombineMode > | p_mode |
Lazy recombination interface | |
| void | setupRecombinationOutput (const DataItem &patchItem, const SharedImageSet *referenceImage) |
| Setup the recombination output image and the weight image. | |
| DataItem | finalizeRecombinationOutput () |
| Recombine a single patch into the output image. | |
Additional Inherited Members | |
| Public Types inherited from Operation | |
| enum | ProcessingPolicy { EverythingExceptLabels = 0 , Everything , OnlyLabels } |
| Policy used by default when selecting on the fields on which the operation will be applied Note that "Labels" here refers to the target tag, not to be confused with Data::Modality::LABEL. More... | |
| using | PreProcessHook = std::function<void(Operation&, DataElement*)> |
| using | PostProcessHook = std::function<void(Operation&, DataElement*)> |
| Static Public Member Functions inherited from Operation | |
| static std::string | processingPolicyToString (const ProcessingPolicy policy) |
| static ProcessingPolicy | stringToProcessingPolicy (const std::string &s) |
| static std::unique_ptr< Operation > | createFromFactories (const Operation::Specs &specs) |
| Helper function to create an operation from any factory (both C++ and Python factories are tested) | |
| Protected Member Functions inherited from Operation | |
| virtual std::shared_ptr< BoundingBoxSet > | processBoxes (std::shared_ptr< BoundingBoxSet > input) const |
| Virtual function that every subclass should override. | |
| virtual std::shared_ptr< KeypointSet > | processPoints (std::shared_ptr< KeypointSet > input) const |
| Virtual function that every subclass should override. | |
| virtual std::shared_ptr< SharedImageSet > | processVectors (std::shared_ptr< SharedImageSet > input) const |
| Virtual function that every subclass should override. | |
| virtual std::shared_ptr< TensorSet > | processTensors (std::shared_ptr< TensorSet > input) const |
| void | throwOperationError (const std::string &msg) const |
| Helper function to throw an exception due to a runtime error. | |
| void | warnOperationUnexpectedBehaviour (const std::string &msg) const |
| Helper function to warn about a behavior different than asked. | |
| bool | inputIsEmptyOrNull (const SharedImageSet *input) const |
| Helper function to check if the input is empty or null and print a warning if so. | |
| virtual bool | useGPU (const SharedImageSet *input) const |
| Helper function that returns which device the operation should choose based on its configuration and a given input. | |
| void | prepareInputForDevice (SharedImageSet &input) const |
| Make sure the input is on the correct device for the current device strategy. | |
| virtual bool | allowChannelBatchOnGPU () const |
| Whether the operation implements channel-batching so that it can still be run on input with more than 4 channels. | |
| bool | configureOnly (const Properties &properties, const std::vector< ParameterBase * > ¶mSelection) noexcept |
| Helper method to configure only a subset of parameters. | |
| void | setManuallyConfiguredParameters (const std::vector< ParameterBase * > &manuallyConfiguredParams) |
| Disable the auto-configuration of some of the parameters so that they can be parsed manually (in the configure function of the derived class). | |
| void | applyPreProcessHooks (DataElement *element) |
| Apply the current pre-process hooks to an element. | |
| void | applyPostProcessHooks (DataElement *element) |
| Apply the current post-process hooks to an element. | |
| Protected Attributes inherited from Operation | |
| std::string | m_name |
| ProcessingPolicy | m_processingFieldsPolicy |
| Flag specifying whether labels should be processed by default (when no active field has been specified) | |
| std::optional< std::unordered_set< std::string > > | m_activeFields |
| Run the Operation only on those fields (if empty, will select suitable fields based on the current processing policy) when calling process() | |
| std::unordered_set< std::string > | m_alreadyWarned |
| Set of warnings about unexpected behavior that have already been printed. | |
| bool | m_errorOnUnexpectedBehaviour = false |
| Whether to throw an exception instead of warning about unexpected behavior. | |
| std::optional< uint32_t > | m_seed |
| Random::Generator | m_randGenerator |
| ML::ComputingDevice | m_device = ML::ComputingDevice::GPUIfOpenGl |
| std::vector< ParameterBase * > | m_params |
| All registered parameters. | |
| std::vector< ParameterBase * > | m_manuallyConfiguredParams |
| All registered parameters that should not be configured automatically. | |
| std::unordered_map< std::string, ElementType > | m_requiredFieldsTypes |
| List of all required field types used by checkRequiredItemsTypes. | |
| std::string | m_recordIdentifier |
| User-provided unique identifier for this operation, used for recording processing history and inversion. | |
| std::vector< PreProcessHook > | m_preProcessHooks |
| std::vector< PostProcessHook > | m_postProcessHooks |
| std::vector< PreProcessHook > | m_temporaryPreProcessHooks |
| Pre-process hooks that will be used for the next processing call only. | |
| std::vector< PostProcessHook > | m_temporaryPostProcessHooks |
| Post-process hooks that will be used for the next processing call only. | |
|
overridevirtual |
Virtual function that every subclass should override.
Default implementation does nothing, so if those function are not implemented in a derived class, they won't act on the input in any way.
Reimplemented from Operation.
|
inlineoverridevirtual |
Returns whether the operation is guaranteed to not modify its input element, either because it computes its output out-of-place or because it is a no-op.
This can be used to decide whether a copy of the input should be made before calling the Operation::process function. Note that this only relates to the overloads taking specific elements as inputs (e.g. images, keypoints, etc.), since the Operation::process(DataItem& item) function always modifies its input.
Reimplemented from Operation.
| void setupRecombinationOutput | ( | const DataItem & | patchItem, |
| const SharedImageSet * | referenceImage ) |
Setup the recombination output image and the weight image.
The reference image is used to determine the size of the output image, while the rest of the image properties (e.g., pixel type, number of channels, etc.) are taken from the patch item.
| DataItem finalizeRecombinationOutput | ( | ) |
Recombine a single patch into the output image.
Also updates the weight image. Before calling this function, setupRecombinationOutput must be called to set up the output image. void recombineSinglePatch(const DataItem& patchItem, const RegionOfInterest& roi);
/ Get the recombined output image. This function normalizes the output image by the weight image cumulated over all patches. / This function should be called after all patches have been recombined via recombineSinglePatch function.
| OpParam<RecombineMode> p_mode |
