#include <ImFusion/Vision/CameraRegistrationAlgorithm.h>

Camera registration algorithm Description: This algorithm performs camera registration for a stereo camera system using known fiducial markers (e.g., chessboard or a ChArUco board) to estimate the relative pose between the left and right cameras. More...

Inheritance diagram for CameraRegistrationAlgorithm:

Detailed Description

Camera registration algorithm Description: This algorithm performs camera registration for a stereo camera system using known fiducial markers (e.g., chessboard or a ChArUco board) to estimate the relative pose between the left and right cameras.

The registration process involves: Detecting markers in synchronized stereo image pairs and Computing the relative transformation(extrinsics) between the stereo cameras

Input:

two image sets (Synchronized) or a stereo image set with Pre-calibrated camera intrinsics
CameraCalibrationSettings as well as markerConfiguration
A StereoSharedImageSet is suggested to use as input

Output:

the result is stored in the input data as a data component (similar to CameraCalibrationAlgorithm)
if the input is a pair of image sets, the registration is store as extrinsics in CameraCalibrationDataComponent in the image set from right camera
if the input is a StereoSharedImageSet, the registration is stored in StereoCalibrationDataComponent

The Algorithm also provide functions to evaluate a extisting registration, e.g computeRMSE()

Public Member Functions
	CameraRegistrationAlgorithm (SharedImageSet img1, SharedImageSet img2)
	Standard constructor.

	CameraRegistrationAlgorithm (StereoSharedImageSet *stereoImg)
	Constructor with stereo shared image set.

SharedImageSet *	inputImages (int idx)

OwningDataList	takeOutput () override
	Returns annotated images, if requested.

void	compute () override
	Runs marker detection and then camera registration.

std::array< std::pair< std::vector< std::vector< vec3 > >, std::vector< std::vector< vec2 > > >, 2 >	computePoints (SharedImageSet out1=nullptr, SharedImageSet out2=nullptr) const
	Computes the relevant points for the given image pairs.

void	setCalibrationSettings (const CameraCalibrationSettings &config)

CameraCalibrationSettings &	calibrationSettings ()

double	rmse () const
	Returns the root mean square reprojection error of the last compute()

void	setMarkerConfig (const MarkerConfiguration &config)

MarkerConfiguration &	markerConfig ()

void	setGenerateDebugImages (bool v)

bool	generatesDebugImages () const

void	configure (const Properties *p) override
	Configure this object instance by de-serializing the given Properties.

void	configuration (Properties *p) const override
	Serialize the current object configuration into the given Properties object.

Public Member Functions inherited from Algorithm
	Algorithm ()
	Default constructor will registers a single "compute" action that calls compute() and returns status().

virtual void	setProgress (Progress *progress)
	Sets a Progress interface the algorithm can use to notify observers about its computing progress.

Progress *	progress () const
	Returns the progress interface if set.

virtual int	status () const
	Indicates the status of the last call to compute().

virtual bool	survivesDataDeletion (const Data *) const
	Indicates whether the algorithm can handle (partial) deletion of the specified data, by default this checks whether the data is in the input list.

const FactoryInfo &	factoryInfo () const
	Returns the record describing how this Algorithm was instantiated by the AlgorithmFactory.

void	setFactoryInfo (const FactoryInfo &value)
	Sets the record describing how this Algorithm was instantiated by the AlgorithmFactory.

Status	runAction (const std::string &id)
	Run the action with name `id` if it exists.

const std::vector< Action > &	actions ()
	Get a mapping from Action id to Action as registered in this algorithm.

Public Member Functions inherited from Configurable
virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.

void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().

void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.

	Configurable (const Configurable &rhs)

	Configurable (Configurable &&rhs) noexcept

Configurable &	operator= (const Configurable &)

Configurable &	operator= (Configurable &&) noexcept

Static Public Member Functions
static bool	createCompatible (const DataList &data, Algorithm **a=0)

static mat4	computeTransform (const std::vector< std::vector< vec3 > > &objectPoints, const std::vector< std::vector< vec2 > > &imagePoints1, const std::vector< std::vector< vec2 > > &imagePoints2, std::array< mat3, 2 > &cameraMatrix, std::array< vec5, 2 > &distortionCoeffs, const std::array< vec2i, 2 > &imageSize, bool optimizeIntrinsics=false, const CameraCalibrationSettings &calibrationSettings=CameraCalibrationSettings())
	Runs registration on given points with given calibration.

static double	computeRMSE (const std::vector< std::vector< vec3 > > &objectPointsPerImagePair, const std::vector< std::vector< vec2 > > &imagePointsPerImage1, const std::vector< std::vector< vec2 > > &imagePointsPerImage2, std::array< mat3, 2 > cameraMatrix, std::array< vec5, 2 > distortionCoeffs, mat4 registrationMatrix)
	(Re)computes the root mean square registration error for a given registration that was loaded from an external source.

Static Public Member Functions inherited from Algorithm
static bool	createCompatible (const DataList &data, Algorithm **a=nullptr)
	Factory function to check algorithm compatibility with input data and optionally instantiate it.

Public Attributes
SubProperty< MarkerConfiguration >	p_markerConfig = {"markerConfiguration", MarkerConfiguration(), *this}

Parameter< bool >	p_optimizeIntrinsics {"optimizeIntrinsics", false, *this}

Parameter< bool >	p_generateDebugImages {"generateDebugImages", false, *this}

Public Attributes inherited from Algorithm
Signal	signalOutputChanged
	Signal should be emitted by Algorithms when their output/result has changed.

Signal	signalParametersChanged
	Signal should be emitted by Algorithms when their parameter configuration has changed.

Public Attributes inherited from Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' `signalValueChanged` signal was emitted.

Additional Inherited Members
Public Types inherited from Algorithm
enum	Status { Unknown = -1 , Success = 0 , Error = 1 , InvalidInput , IncompleteInput , OutOfMemoryHost , OutOfMemoryGPU , UnsupportedGPU , UnknownAction , AbortedByUser , User = 1000 }
	Status codes. More...

Protected Member Functions inherited from Algorithm
void	loadDefaults ()

void	registerAction (const std::string &id, const std::string &guiName, const std::function< Algorithm::Status(void)> &action)
	Register an action to be run via runAction.

template<typename D>
void	registerAction (const std::string &id, const std::string &guiName, Algorithm::Status(D::*action)(void))
	Template version of runAction that can be used with a pointer to a member function.

void	registerAction (const Action &action)
	Register an action.

Protected Attributes inherited from Algorithm
std::string	m_name
	Algorithm name.

Progress *	m_progress = nullptr
	Non-owing pointer to a progress interface. May be a nullptr.

FactoryInfo	m_factoryInfo = {}
	Record describing how this algorithm was instantiated by the AlgorithmFactory.

int	m_status = Status::Unknown
	Algorithm status after last call to compute()

std::vector< Action >	m_actions
	Map of key given by the id of the action, of the available actions of this algorithm.

Protected Attributes inherited from Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Constructor & Destructor Documentation

◆ CameraRegistrationAlgorithm() [1/2]

CameraRegistrationAlgorithm	(	SharedImageSet *	img1,
		SharedImageSet *	img2 )

Standard constructor.

Parameters

[in] img1,img2 two image sets containing multiple images of a calibration target from two cameras

◆ CameraRegistrationAlgorithm() [2/2]

CameraRegistrationAlgorithm ( StereoSharedImageSet * stereoImg )

Constructor with stereo shared image set.

Parameters

[in] stereoImg stereo image sets containing a pair of multiple images sets of a calibration target from two cameras

Member Function Documentation

◆ takeOutput()

OwningDataList takeOutput ( )

overridevirtual

Returns annotated images, if requested.

If debug images were generated (see setGenerateDebugImages()), the input images are annotated with the detected points. Those annotated images can be retrieved here.

Returns: An OwningDataList with two SharedImageSets, each containing the annotated input images

Reimplemented from Algorithm.

◆ compute()

void compute ( )

overridevirtual

Runs marker detection and then camera registration.

If successful, this method also sets the internal rmse property, which can be queried by rmse(), and sets the registration matrices of the images' CameraCalibrationDataComponent so that both map to the system of camera 1: camea 1 gets the identity matrix, camera 2 gets the inverse matrix of registraion().

Implements Algorithm.

◆ computePoints()

std::array< std::pair< std::vector< std::vector< vec3 > >, std::vector< std::vector< vec2 > > >, 2 > computePoints	(	SharedImageSet *	out1 = nullptr,
		SharedImageSet *	out2 = nullptr ) const

Computes the relevant points for the given image pairs.

This method computes object points and image points for each image and input image set.

Note: Some images may be filtered out due to internal constraints and by checking for p_minDetections (

See also: minDetections()), so the mapping from returned points to images is not 1:1.

Parameters

[out] out1,out2 (optional) SharedImageSets that get annotated with the detected points. If given, the SharedImageSets must have the same number of images with the same size as the input images.

Returns: An array which contains a pair of (object points, image points) per input image set. Each pair contains a vector of vectors, containing points for images.

◆ computeTransform()

static mat4 computeTransform	(	const std::vector< std::vector< vec3 > > &	objectPoints,
		const std::vector< std::vector< vec2 > > &	imagePoints1,
		const std::vector< std::vector< vec2 > > &	imagePoints2,
		std::array< mat3, 2 > &	cameraMatrix,
		std::array< vec5, 2 > &	distortionCoeffs,
		const std::array< vec2i, 2 > &	imageSize,
		bool	optimizeIntrinsics = false,
		const CameraCalibrationSettings &	calibrationSettings = CameraCalibrationSettings() )

static

Runs registration on given points with given calibration.

The passed arrays need to contain the same number of points for each image.

Parameters

[in]	objectPoints	Coordinates of mutually detected points for each image pair in world coordinates
[in]	imagePoints1	Coordinates of mutually detected points for each image pair in the first image
[in]	imagePoints2	Coordinates of mutually detected points for each image pair in the second image
[in,out]	cameraMatrix	Camera intrinsic matrices for the respective imagePoints. Contains optimized intrinsics on return if intrinsics optimization is activated.
[in,out]	distortionCoeffs	Camera distortion coefficients for the respective imagePoints. Contains optimized distortion paramters on return if intrinsics optimization is activated.
[in]	imageSize	Camera image sizes
[in]	optimizeIntrinsics	Specifies whether camera intrinsics are optimized together with stereo registration
[in]	calibrationSettings	Calibration settings used when optimizeIntrinsics is set

Returns: The computed registration from camera 1 to camera 2, e.g. the transformation from the first camera coordinate system to the second camera coordinate system

◆ computeRMSE()

static double computeRMSE	(	const std::vector< std::vector< vec3 > > &	objectPointsPerImagePair,
		const std::vector< std::vector< vec2 > > &	imagePointsPerImage1,
		const std::vector< std::vector< vec2 > > &	imagePointsPerImage2,
		std::array< mat3, 2 >	cameraMatrix,
		std::array< vec5, 2 >	distortionCoeffs,
		mat4	registrationMatrix )

static

(Re)computes the root mean square registration error for a given registration that was loaded from an external source.

Object points and image points respecting the current algorithm parameters can be obtained by computePoints().

Parameters

[in]	objectPointsPerImagePair
[in]	imagePointsPerImage1
[in]	imagePointsPerImage2
[in]	cameraMatrix	Camera matrix per side, if available. If none is given, computeRMSE will try to take it from the images' CameraCalibrationDataComponent
[in]	distortionCoeffs	Distortion coefficients per side, if available. If none is given, computeRMSE will try to take it from the images' CameraCalibrationDataComponent
[in]	registrationMatrix	Matrix that brings points given in the first camera's coordinate system to points in the second camera's coordinate system

Returns: the computed RMSE.

◆ rmse()

double rmse ( ) const

Returns the root mean square reprojection error of the last compute()

Note: Only compute() will update this internal state. computeTransform() and/or computeRMSE() do not set this.

Returns: Root mean square reprojection error

◆ configure()

void configure ( const Properties * p )

overridevirtual

Configure this object instance by de-serializing the given Properties.

The default implementation will do so automatically for all registered Parameter and SubProperty instances.

See also: configuration() for the inverse functionality

Reimplemented from Configurable.

◆ configuration()

void configuration ( Properties * p ) const

overridevirtual

Serialize the current object configuration into the given Properties object.

The default implementation will do so automatically for all registered Parameter and SubProperty instances.

See also: configure() for the inverse functionality

Reimplemented from Configurable.

The documentation for this class was generated from the following file:

ImFusion/Vision/CameraRegistrationAlgorithm.h

Detailed Description

Public Member Functions

Static Public Member Functions

Public Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ CameraRegistrationAlgorithm() [1/2]

◆ CameraRegistrationAlgorithm() [2/2]

Member Function Documentation

◆ takeOutput()

◆ compute()

◆ computePoints()

◆ computeTransform()

◆ computeRMSE()

◆ rmse()

◆ configure()

◆ configuration()