#include <ImFusion/Stream/LiveStreamProcessingAlgorithm.h>

Algorithm for live streams: merges ImageStreamData and TrackingStreamData from connected Stream instances into an internal buffer on stream threads, then promotes to a live TrackedSharedImageSet when the host calls compute(). More...

Inheritance diagram for ImFusion::LiveStreamProcessingAlgorithm:

Detailed Description

Algorithm for live streams: merges ImageStreamData and TrackingStreamData from connected Stream instances into an internal buffer on stream threads, then promotes to a live TrackedSharedImageSet when the host calls compute().

Recording the stream to disk is unrelated to this class.

You may construct this class directly for the default live output type, or subclass to change that type, how data is published to the live output, when refresh runs, or to use an optional background worker. If the constructor requests a background worker, this class prepares an extra buffer for results produced off the UI thread; filling that buffer is up to subclasses—this implementation does not write it.

LiveStreamProcessingAlgorithm alg(std::vector<Stream*>{ imageStream, trackingStream });
alg.createLiveContainer();
alg.start();
// Connect Algorithm::signalOutputChanged so the main thread calls compute() when data changes (as controllers do).
alg.compute();
OwningDataList out = alg.takeOutput();

Packet types other than image or tracking are skipped. Algorithm::signalOutputChanged indicates the live output may be stale; run compute() on the main thread before relying on what is on screen.

Intended for live imaging together with tracking. Other kinds of stream content need a different entry point unless you extend how packets are handled.

Data from the streams is merged on those threads into an internal buffer. When the live picture should catch up, run compute() on the main thread so the TrackedSharedImageSet matches that merged state.

With no background worker, compute() copies from that buffer into the live output. With a background worker, heavier work can finish in a separate buffer first, and compute() then copies from there; subclasses supply what appears in that second buffer. Tracking-only changes still ask for a UI refresh; new images are what prompts the background worker to run again.

Note: For customization, see createLiveContainer(), compute(), moveBufferedDataToLiveContainer(), onNewImagesArrived(), and backgroundThreadProcessData().

Public Member Functions
	LiveStreamProcessingAlgorithm (std::vector< Stream * > inputStreams, bool hasBackgroundThread=false, bool createOpenGLContextInProcessingThread=false)
void	compute () override
	Moves the data from the back buffer to the live container, accessible through the output interface.
OwningDataList	takeOutput () override
	Transfers the live container and sets m_ownLiveContainer to false.
virtual void	createLiveContainer ()
	Fills m_liveContainer with a suitable container, default implementation creates a new TrackedSharedImageSet.
virtual void	start ()
	Sets m_isRunning to true.
virtual void	stop ()
	Sets m_isRunning to false.
bool	isRunning () const
	Returns m_isRunning.
Public Member Functions inherited from ImFusion::Algorithm
	Algorithm ()
	Default constructor will registers a single "compute" action that calls compute() and returns status().
virtual void	setProgress (Progress *progress)
	Sets a Progress interface the algorithm can use to notify observers about its computing progress.
Progress *	progress () const
	Returns the progress interface if set.
virtual int	status () const
	Indicates the status of the last call to compute().
virtual bool	survivesDataDeletion (const Data *) const
	Indicates whether the algorithm can handle (partial) deletion of the specified data, by default this checks whether the data is in the input list.
const FactoryInfo &	factoryInfo () const
	Returns the record describing how this Algorithm was instantiated by the AlgorithmFactory.
void	setFactoryInfo (const FactoryInfo &value)
	Sets the record describing how this Algorithm was instantiated by the AlgorithmFactory.
Status	runAction (const std::string &id)
	Run the action with name id if it exists.
const std::vector< Action > &	actions ()
	Get a mapping from Action id to Action as registered in this algorithm.
Public Member Functions inherited from ImFusion::Configurable
virtual void	configure (const Properties *p)
	Configure this object instance by de-serializing the given Properties.
virtual void	configuration (Properties *p) const
	Serialize the current object configuration into the given Properties object.
virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.
void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().
void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.
	Configurable (const Configurable &rhs)
	Configurable (Configurable &&rhs) noexcept
Configurable &	operator= (const Configurable &)
Configurable &	operator= (Configurable &&) noexcept
Public Member Functions inherited from ImFusion::SignalReceiver
	SignalReceiver ()=default
	Default constructor.
	SignalReceiver (const SignalReceiver &other)
	Copy constructor, does not copy any existing signal connections from other.
SignalReceiver &	operator= (SignalReceiver rhs)
	Assignment operator, disconnects all existing connections, does not copy any existing signal connections from rhs.
virtual	~SignalReceiver ()
	Virtual destructor disconnects from all connected signals.

Public Attributes
Parameter< bool >	p_useDeviceTimestamp
	If true, buffered samples use device timestamps; otherwise arrival time.
Parameter< std::optional< int > >	p_imageBufferSize {"imageBufferSize", std::nullopt, this}
	Pre-buffer image cap; nullopt = unlimited.
Parameter< std::optional< int > >	p_trackingBufferSize {"trackingBufferSize", std::nullopt, this}
	Pre-buffer cap per tracking sequence; nullopt = unlimited.
Parameter< double >	p_qualityThreshold {"qualityThreshold", 0.2, this}
	Quality threshold for matrix updates.
Public Attributes inherited from ImFusion::Algorithm
Signal	signalOutputChanged
	Signal should be emitted by Algorithms when their output/result has changed.
Signal	signalParametersChanged
	Signal should be emitted by Algorithms when their parameter configuration has changed.
Public Attributes inherited from ImFusion::Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' signalValueChanged signal was emitted.

Protected Member Functions
void	onNewStreamData (std::shared_ptr< const StreamData > streamData)
	Temporarily buffers copies of the streamData in internal queues.
virtual void	onNewImagesArrived ()
	Announces either to the main thread (no background thread) or the background processing thread that new images have arrived.
void	notifyListenersOutputChanged ()
	Emits Algorithm::signalOutputChanged so listeners can schedule a main-thread update (e.g. compute()).
double	extractTimestamp (const StreamData *data) const
	Extracts timestampDevice() or timestampArrival(), depending on p_useDeviceTimestamp.
void	startProcessingThread ()
	Starts the processing thread, which will call backgroundThreadProcessData() after each notification.
void	stopProcessingThread ()
	Stops the processing thread.
virtual bool	backgroundThreadProcessData ()
	This method is called from the processing thread after each notification. To be called again without sleeping if returned true.
void	moveImages (TrackedSharedImageSet tsisSrc, TrackedSharedImageSet tsisDst)
	Moves all images from the source to the destination container.
void	ensureSufficientTracking (TrackedSharedImageSet tsisSrc, TrackedSharedImageSet tsisDst)
	Creates tracking streams as necessary for moveTracking() to be able to copy all tracking data.
void	moveTracking (TrackedSharedImageSet tsisSrc, TrackedSharedImageSet tsisDst)
	Moves all tracking data from the source to the destination container.
void	moveImagesAndTracking (TrackedSharedImageSet tsisSrc, TrackedSharedImageSet tsisDst)
	Moves all images and tracking data from the source to the destination container.
virtual void	moveBufferedDataToLiveContainer ()
	Moves images and tracking data from one of the data buffers (depending on whether there is a background processing thread).
void	selectAllFocusLast ()
	Helper method to select all frames of the live container and focus on the last one.
Protected Member Functions inherited from ImFusion::Algorithm
void	loadDefaults ()
void	registerAction (const std::string &id, const std::string &guiName, const std::function< Algorithm::Status(void)> &action)
	Register an action to be run via runAction.
template<typename D>
void	registerAction (const std::string &id, const std::string &guiName, Algorithm::Status(D::*action)(void))
	Template version of runAction that can be used with a pointer to a member function.
void	registerAction (const Action &action)
	Register an action.
Protected Member Functions inherited from ImFusion::SignalReceiver
void	disconnectAll ()
	Disconnects all existing connections.

Protected Attributes
std::vector< Stream * >	m_inputStreams
	Input.
const bool	m_hasBackgroundThread
	Flag whether there is a background thread.
TrackedSharedImageSet *	m_liveContainer = nullptr
	Output Live container, see m_ownLiveContainer for ownership.
bool	m_ownLiveContainer = false
	If true, the live container is still owned by this algorithm; false after output() has transferred it.
std::unique_ptr< TrackedSharedImageSet >	m_dataBufferPre
	Staging buffer between onNewStreamData and further processing / live container.
std::vector< GL::SyncObject >	m_dataBufferPreGlFences
	Fences to synchronize the processing thread OpenGL context with incoming image streams' GL contexts.
std::mutex	m_dataBufferPreMutex
	Mutex protecting m_dataBufferPre and m_dataBufferPreGlFences.
std::map< std::string, mat4 >	m_lastGoodPositions
	Cache for last matrix of each instrument with good quality.
std::unique_ptr< TrackedSharedImageSet >	m_dataBufferPost
	Second staging container; filled by derived logic when using the background path.
std::mutex	m_dataBufferPostMutex
	Mutex protecting m_dataBufferPost.
std::atomic< bool >	m_isRunning = {false}
	Flag whether stream events are being processed.
std::future< void >	m_processingThread
	Background thread.
std::mutex	m_processingThreadMutex
	Used with m_processingThreadCondition and the background worker loop.
bool	m_processingThreadStopExecution
	Flag indicating the background thread to stop.
std::condition_variable	m_processingThreadCondition
	Cond.var. to wake up background thread.
bool	m_createOpenGLContextInProcessingThread = false
	If true, worker thread creates and owns one shared GL context.
Protected Attributes inherited from ImFusion::Algorithm
std::string	m_name
	Algorithm name.
Progress *	m_progress = nullptr
	Non-owing pointer to a progress interface. May be a nullptr.
FactoryInfo	m_factoryInfo = {}
	Record describing how this algorithm was instantiated by the AlgorithmFactory.
int	m_status = Status::Unknown
	Algorithm status after last call to compute().
std::vector< Action >	m_actions
	Map of key given by the id of the action, of the available actions of this algorithm.
Protected Attributes inherited from ImFusion::Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Additional Inherited Members
Public Types inherited from ImFusion::Algorithm
enum	Status { Unknown = -1 , Success = 0 , Error = 1 , InvalidInput , IncompleteInput , OutOfMemoryHost , OutOfMemoryGPU , UnsupportedGPU , UnknownAction , AbortedByUser , User = 1000 }
	Status codes. More...
Static Public Member Functions inherited from ImFusion::Algorithm
static bool	createCompatible (const DataList &data, Algorithm **a=nullptr)
	Factory function to check algorithm compatibility with input data and optionally instantiate it.

Member Function Documentation

◆ compute()

void ImFusion::LiveStreamProcessingAlgorithm::compute ( )

overridevirtual

Moves the data from the back buffer to the live container, accessible through the output interface.

Implements ImFusion::Algorithm.

Reimplemented in ImFusion::TrackingStreamVisualizationAlgorithm, and ImFusion::US::LiveSweepRecordingVisualizationAlgorithm.

◆ takeOutput()

OwningDataList ImFusion::LiveStreamProcessingAlgorithm::takeOutput ( )

overridevirtual

Transfers the live container and sets m_ownLiveContainer to false.

Reimplemented from ImFusion::Algorithm.

◆ createLiveContainer()

virtual void ImFusion::LiveStreamProcessingAlgorithm::createLiveContainer ( )

virtual

Fills m_liveContainer with a suitable container, default implementation creates a new TrackedSharedImageSet.

Reimplemented in ImFusion::US::LiveSweepRecordingVisualizationAlgorithm.

◆ moveImages()

void ImFusion::LiveStreamProcessingAlgorithm::moveImages	(	TrackedSharedImageSet *	tsisSrc,
		TrackedSharedImageSet *	tsisDst )

protected

Moves all images from the source to the destination container.

After the operation, the source container does not contain any images.

◆ moveTracking()

void ImFusion::LiveStreamProcessingAlgorithm::moveTracking	(	TrackedSharedImageSet *	tsisSrc,
		TrackedSharedImageSet *	tsisDst )

protected

Moves all tracking data from the source to the destination container.

Creates tracking streams as necessary. After the operation, all the source container's tracking streams still exist but are empty.

◆ moveImagesAndTracking()

void ImFusion::LiveStreamProcessingAlgorithm::moveImagesAndTracking	(	TrackedSharedImageSet *	tsisSrc,
		TrackedSharedImageSet *	tsisDst )

protected

Moves all images and tracking data from the source to the destination container.

Creates tracking streams as necessary. After the operation, the source container does not contain any images, and all its tracking streams are empty.

◆ moveBufferedDataToLiveContainer()

virtual void ImFusion::LiveStreamProcessingAlgorithm::moveBufferedDataToLiveContainer ( )

protectedvirtual

Moves images and tracking data from one of the data buffers (depending on whether there is a background processing thread).

Should only be called from the main thread if the live container is already shown in any view.

Reimplemented in ImFusion::US::LiveSweepRecordingVisualizationAlgorithm.

Member Data Documentation

◆ p_useDeviceTimestamp

Parameter<bool> ImFusion::LiveStreamProcessingAlgorithm::p_useDeviceTimestamp

Initial value:

{"useDeviceTimestamps",
                                             false,
                                             this}

If true, buffered samples use device timestamps; otherwise arrival time.

◆ m_inputStreams

std::vector<Stream*> ImFusion::LiveStreamProcessingAlgorithm::m_inputStreams

protected

Input.

Input streams listened to

◆ m_liveContainer

TrackedSharedImageSet* ImFusion::LiveStreamProcessingAlgorithm::m_liveContainer = nullptr

protected

Output Live container, see m_ownLiveContainer for ownership.

Is returned by output() in each call.

◆ m_processingThread

std::future<void> ImFusion::LiveStreamProcessingAlgorithm::m_processingThread

protected

Background thread.

Background processing thread future

The documentation for this class was generated from the following file:

ImFusion/Stream/LiveStreamProcessingAlgorithm.h

Detailed Description

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Additional Inherited Members

Member Function Documentation

◆ compute()

◆ takeOutput()

◆ createLiveContainer()

◆ moveImages()

◆ moveTracking()

◆ moveImagesAndTracking()

◆ moveBufferedDataToLiveContainer()

Member Data Documentation

◆ p_useDeviceTimestamp

◆ m_inputStreams

◆ m_liveContainer

◆ m_processingThread