#include <RGBD/Include/ImFusion/RGBD/RGBDPlayback.h>

Playback of stored RGB-D sequences. More...

Inheritance diagram for RGBDPlayback:

Detailed Description

Playback of stored RGB-D sequences.

This class provides playback functionality for previously recorded RGB-D data, supporting frame stepping, looping, and selection of frames and sensors. It wraps an RGBDMultiSequence and exposes the interface of an RGBDStream, making it compatible with streaming and processing workflows.

Usage:

Construct with a unique pointer to an RGBDMultiSequence.
Control playback using reset(), singleStep(), replayFrame(), and looping options.
Access current frame, frame rate, and sensor parameters.
Use selection and focus features to restrict playback to specific frames or sensors.

Important aspects:

Supports two looping modes: MirrorDirection and StartOver.
Playback can be limited to selected frames.
Provides signals for frame changes.
Thread safety is partially ensured via internal mutex.

Example:

std::unique_ptr<RGBDMultiSequence> multiSeq;
auto playback = std::make_unique<RGBDPlayback>(std::move(multiSeq));
playback->setFrameRate(25.0);
playback->setLoop(true);
 
// Connect to the playback's signal signalStreamData here to get the frames
 
playback->start();
    
// Do something while the playback is running
while (playback->isRunning())
{
}   

Public Types
enum	LoopingMode { MirrorDirection = 0 , StartOver = 1 }
	Looping mode for playback. If looping is enabled, this determines how playback continues after reaching the end. More...

Public Types inherited from RGBDStream
enum	PixelFormat { Rgb24 , Rgb32 , Depth16 , Gray8 , Gray16 , Yuv422 }
	Pixel format enumeration for RGB-D frames. More...

Public Types inherited from Stream
enum class	ThreadingMode { BaseClassRunsWorkerThread , DerivedClassHandlesThreading }
	This flag represents the two supported threading modes of a Stream. More...

enum class	State { Closed , Opening , Open , Starting , Running , Pausing , Paused , Resuming , Stopping , Closing }

Public Types inherited from Data
enum	Kind { UNKNOWN = 0 , IMAGE = 1 , VOLUME = 2 , IMAGESET = 3 , VOLUMESET = 4 , IMAGESTREAM = 5 , VOLUMESTREAM = 6 , POINTSET = 7 , SURFACE = 8 , TRACKINGSTREAM = 9 , LIVETRACKINGSTREAM = TRACKINGSTREAM , TRACKINGDATA = 10 , TREE = 11 , TENSOR = 12 , POLYDATASTREAM = 13 , STEREOIMAGESET = 14 , STEREOIMAGESTREAM = 15 , VOLUMETRICMESH = 16 }
	Kind of data. More...

enum	Modality { NA = 0 , XRAY = 1 , CT = 2 , MRI = 3 , ULTRASOUND = 4 , VIDEO = 5 , NM = 6 , OCT = 7 , LABEL = 8 , DISTANCE = 9 }
	Image modality of the data. More...

Public Member Functions
	RGBDPlayback (std::unique_ptr< RGBDMultiSequence > pbData)
	Construct playback from a multi-sequence (takes ownership).

void	setPlaybackData (std::unique_ptr< RGBDMultiSequence > pbData)
	Set new playback data (takes ownership).

RGBDMultiSequence *	playbackData () const
	Get the current playback data. Does not transfer ownership.

bool	reset () override
	Stop playback and rewind to beginning.

bool	singleStep ()
	Play exactly one frame at current position and then pause.

bool	replayFrame ()
	Play frame at current position again and then pause.

void	setFrameRate (double fps)
	Set playback frame rate.

double	frameRate () const
	Get playback frame rate.

void	setStopFrame (int frame)
	Set the frame at which playback should stop.

int	stopFrame () const
	Get the stop frame.

void	setLoop (bool loop)
	Enable or disable looping.

bool	loop () const
	Returns whether looping is enabled.

void	setLoopingMode (LoopingMode mode)
	Set the looping mode. Only has an effect if looping is enabled.

bool	loopingMode () const
	Get the current looping mode.

int	currentFrame () const
	Get the current frame index.

int	currentFrame (int sensor) const
	Get the current frame index for a specific sensor.

void	setCurrentFrame (int frame)
	Set the current frame index.

bool	depthK (mat3 &K) const override
	Get depth camera intrinsics.

bool	colorK (mat3 &K) const override
	Get color camera intrinsics.

bool	depthToColorT (mat4 &T) const override
	Get transformation from depth to color camera.

mat4	matrix () const override
	Get current transformation matrix.

void	setMatrix (const mat4 &m) override
	Set current transformation matrix.

bool	supportsColor () const override
	Returns true if color images are supported.

std::string	uuid () override
	Get the UUID of the playback stream.

int	depthImageWidth () const override
	Get depth image width.

int	depthImageHeight () const override
	Get depth image height.

int	colorImageWidth () const override
	Get color image width.

int	colorImageHeight () const override
	Get color image height.

int	currentSensor () const
	Get the current sensor index.

std::string	describe () const override
	Get a human-readable description of the playback stream.

void	setSelection (const Selection &sel) override
	Set the selection of frames/sensors for playback.

void	setFocus (int index) override
	Set the focus to a specific sensor.

int	size () const override
	Get the number of sensors/sequences.

void	setPlaySelected (bool set)
	If set true, only reads/streams frames selected.

bool	playSelected () const
	Returns if only the selected frames are read/streamed.

virtual std::string	serialNumber () const override
	Get the serial number of the device.

Public Member Functions inherited from RGBDStream
virtual bool	depthFormats (std::vector< FrameInfo > &formats)

virtual bool	colorFormats (std::vector< FrameInfo > &formats)

virtual bool	depthFormat (FrameInfo &format) const

virtual bool	colorFormat (FrameInfo &format) const

virtual bool	setDepthFormat (const FrameInfo &format)

virtual bool	setColorFormat (const FrameInfo &format)

virtual bool	grabSingleFrame ()
	Grabs a single RGBD frame.

virtual bool	isOpen () const

virtual std::string	name () const

virtual std::string	firmwareVersion () const

virtual bool	isExposureSupported () const

virtual bool	setExposure (int value)

virtual int	exposure () const

	RGBDStream (const std::string &name="")

virtual bool	supportsAutoWhiteBalance () const

virtual bool	setAutoWhiteBalanceEnabled (bool enable)

virtual bool	isAutoWhiteBalanceEnabled () const

virtual bool	supportsAutoExposure () const

virtual bool	setAutoExposureEnabled (bool enable)

virtual bool	isAutoExposureEnabled () const

virtual bool	isGainSupported () const

virtual bool	setGain (int value)

virtual int	gain () const

virtual bool	grabImage (float depthData, unsigned char colorData, int timeOut=500)

virtual bool	usingFactoryCalibration () const
	Returns true when the sensor has a per-sensor factory calibration and it is currently being used.

virtual bool	depthDist (Eigen::Matrix< double, 5, 1 > &dist) const
	Retrieve depth camera distortion parameters if available.

virtual bool	dropCorruptDepthData (bool &drop) const

virtual bool	setDropCorruptDepthData (bool drop)

virtual bool	colorDist (Eigen::Matrix< double, 5, 1 > &dist) const
	Retrieve color camera distortion parameters if available.

virtual bool	setDepthK (const mat3 &K)

virtual bool	setDepthDist (const Eigen::Matrix< double, 5, 1 > &dist)

virtual bool	setColorK (const mat3 &K)

virtual bool	setColorDist (const Eigen::Matrix< double, 5, 1 > &dist)

virtual bool	setDepthToColorT (const mat4 &T)

virtual bool	setLaserState (bool enable)

virtual bool	laserState () const

virtual bool	supportsSingleFrameGrab () const
	Returns if the stream supports single frame capture model; set false by default.

virtual bool	startOnImport () const
	Returns if the stream shall be started on import; set true by default.

virtual bool	loadCalibration (const Properties *p)

virtual bool	saveCalibration (Properties *p) const

virtual bool	selectDefaultFrameFormat ()

virtual bool	createDefaultStreamController () override final
	Indicates whether a StreamController should be created for the stream.

std::unique_ptr< RGBDDataComponent >	getRGBDDataComponentCopy ()
	Returns the copy of the RGBDDataComponent. If it wasn't present before, it is added and updated.

Public Member Functions inherited from ImageStream
	ImageStream (const std::string &name="")

Data::Kind	kind () const override final
	Returns the kind (typically IMAGESTREAM or VOLUMESTREAM)

void	setModality (Modality m)
	Set the modality descriptor of this stream.

Modality	modality () const override final
	Return the modality of this stream.

virtual bool	topDown () const
	Specify whether the first line of the image data is to be displayed on top.

Public Member Functions inherited from Stream
	Stream (const std::string &name="")
	Constructor with the `name` for this stream.

virtual ThreadingMode	threadingMode () const
	Configures threading mode of stream.

virtual bool	supportsAsyncOperation ()
	Returns true if xxxAsync() methods support execution in a background thread.

std::shared_future< bool >	openAsync ()
	Open stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

std::shared_future< bool >	closeAsync ()
	Close stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

std::shared_future< bool >	startAsync ()
	Start stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

std::shared_future< bool >	stopAsync ()
	Stop stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

std::shared_future< bool >	pauseAsync ()
	Pause stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

std::shared_future< bool >	resumeAsync ()
	Resume stream in an asynchronous fashion (wait on the returned future before taking further actions). Returns whether the operation was successful.

bool	open ()
	Open stream and wait for the operation to complete. Returns whether the operation was successful.

bool	close ()
	Close stream and wait for the operation to complete. Returns whether the operation was successful.

bool	start ()
	Start stream and wait for the operation to complete. Returns whether the operation was successful.

bool	stop ()
	Stop stream and wait for the operation to complete. Returns whether the operation was successful.

bool	pause ()
	Pause stream and wait for the operation to complete. Returns whether the operation was successful.

bool	resume ()
	Resume stream and wait for the operation to complete. Returns whether the operation was successful.

virtual bool	supportsPausing () const
	Override and return true if this stream implements the pausing and resume operations.

bool	restart ()
	Stops, closes, opens and starts the stream. Returns whether the operation was successful.

virtual bool	isRunning () const
	Return whether stream is currently running.

State	currentState () const

bool	isStateOneOf (const std::vector< State > &states) const

Geometry::AlignedBox	bounds () const override
	Per default a Stream has empty/invalid bounds.

Pose::TransformationConvention	matrixConvention () const override
	Matrices of streams by default map to the world coordinate system.

double	updateRate () const
	Average update rate in Hz of the `signalStreamData`.

std::string	describe () const override
	Short description of the stream including latest state and update rate information.

Public Member Functions inherited from Data
	Data (const std::string &name="")

	Data (const Data &other)

Data &	operator= (const Data &other)

virtual	~Data ()
	Mandatory virtual destructor.

const std::string &	name () const
	Return the name of this data.

void	setName (const std::string &name)
	Sets the name of this data.

virtual bool	isAnnotationType () const
	Return whether this data type is visualized through an annotation (e.g. mesh)

const DataComponentList &	components () const
	Returns the list of DataComponents for this data.

DataComponentList &	components ()

virtual void	setMatrixFromWorld (const mat4 &m)
	Set matrix mapping from the world coordinate system to the data coordinate system.

virtual void	setMatrixToWorld (const mat4 &m)
	Set matrix mapping from the data coordinate system to the world coordinate system.

virtual mat4	matrixFromWorld () const
	Get matrix mapping from the world coordinate system to the data coordinate system.

virtual mat4	matrixToWorld () const
	Get matrix mapping from the data coordinate system to the world coordinate system.

Public Member Functions inherited from Configurable
virtual void	configure (const Properties *p)
	Configure this object instance by de-serializing the given Properties.

virtual void	configuration (Properties *p) const
	Serialize the current object configuration into the given Properties object.

virtual void	configureDefaults ()
	Retrieve the properties of this object, replaces values with their defaults and sets it again.

void	registerParameter (ParameterBase *param)
	Register the given Parameter or SubProperty, so that it will be configured during configure()/configuration().

void	unregisterParameter (const ParameterBase *param)
	Remove the given Parameter or SubProperty from the list of registered parameters.

	Configurable (const Configurable &rhs)

	Configurable (Configurable &&rhs) noexcept

Configurable &	operator= (const Configurable &)

Configurable &	operator= (Configurable &&) noexcept

Public Member Functions inherited from Selectable
	Selectable (const Selectable &)

	Selectable (Selectable &&)

Selectable &	operator= (const Selectable &)

Selectable &	operator= (Selectable &&)

virtual Selection	selection () const

virtual int	focus () const

Public Attributes
Signal< int >	signalChangedFrame
	Signal emitted when the frame changes.

Public Attributes inherited from ImageStream
Signal< Data::Kind >	signalKindChanged
	Is emitted when the stream changes between IMAGESTREAM and VOLUMESTREAM, before the next ImageStreamData is emitted.

Public Attributes inherited from Stream
ProtectedSignal< StateChange >	signalStateChanged
	Emitted after a state change has been completed with the old and new state.

ProtectedSignal	signalEnded
	Indicates that the stream reached the end of its data, e.g. when a video stream has played to the end.

ProtectedSignal< std::shared_ptr< const StreamData > >	signalStreamData
	Signal that is emitted by the stream when a new StreamData is available.

DeprecatedSignal< ProtectedSignal< std::shared_ptr< const StreamData > >, const StreamData & >	signalNewData
	Signal that is emitted by the stream when a new StreamData is available.

Public Attributes inherited from Data
Signal< const Data * >	signalDeleted
	Signal emitted when this instance is deleted.

Signal< const Data * >	signalMatrixChanged
	Signal emitted when the transformation of this Data has changed.

Signal< std::string >	signalNameChanged
	Signal emitted when the name changed.

Public Attributes inherited from Configurable
Signal	signalParametersChanged
	Emitted whenever one of the registered Parameters' or SubPropertys' `signalValueChanged` signal was emitted.

Public Attributes inherited from Selectable
ProtectedSignal	signalSelectionChanged
	Signals that the selection or size might have changed.

Protected Member Functions
void	onRGBDDataComponentChanged ()
	Called when the RGBDDataComponent changes.

bool	openImpl () override
	Load recording. Returns false when loading failed.

bool	closeImpl () override
	Close stream.

bool	startImpl () override
	Start playback of loaded file.

bool	stopImpl () override
	Pause playback at current position.

std::optional< WorkContinuation >	doWork () override
	Perform playback work (internal).

Protected Member Functions inherited from RGBDStream
virtual bool	defaultColorResolution (int &width, int &height)
	Returns default color resolution.

virtual bool	defaultDepthResolution (int &width, int &height)
	Returns default depth resolution.

Protected Member Functions inherited from ImageStream
void	setKind (Kind k)
	Sets the kind (typically IMAGESTREAM or VOLUMESTREAM) and emits signalKindChanged.

bool	workerThreadRequiresOpenGl () const override
	Ensure the thread has an GL context before the main loop starts.

Protected Member Functions inherited from Stream
virtual bool	pauseImpl ()
	Pause stream (not required to be supported).

virtual bool	resumeImpl ()
	Resume stream from pause (only required to be supported when pauseImpl() is supported).

virtual bool	worksWhilePaused () const
	Override if doWork() should be called also in the Paused state.

void	changeState (State newState)

std::shared_future< bool >	attemptStateChange (std::function< bool()> work, State stateWorking, State stateSuccess, State stateError)

Protected Member Functions inherited from Data
void	swapWith (Data &other)
	Swaps the data and emits a matrix and name changed signal for both.

Additional Inherited Members
Static Public Member Functions inherited from Stream
static std::string	stateToString (State state)

Static Public Member Functions inherited from Data
static std::string	modalityString (Data::Modality m)
	Return the name of an image modality.

static Data::Modality	stringToModality (const std::string &s)
	Returns the modality corresponding to a modality string or NA if nothing matches.

Protected Attributes inherited from Data
mat4	m_matrix
	Transformation matrix.

std::recursive_mutex *	m_matrixMutex
	Used to internally synchronize access to the matrix of data.

DataComponentList	m_dataComponentList
	The list of DataComponents for this data.

Protected Attributes inherited from Configurable
std::vector< Param >	m_params
	List of all registered Parameter and SubProperty instances.

Protected Attributes inherited from Selectable
std::mutex	m_selectionMutex
	Mutex protecting the selection.

Selection	m_sel

Member Enumeration Documentation

◆ LoopingMode

enum LoopingMode

Looping mode for playback. If looping is enabled, this determines how playback continues after reaching the end.

Enumerator
MirrorDirection	When ended, goes from last to first.
StartOver	When ended, starts from the first.

Member Function Documentation

◆ reset()

bool reset ( )

overridevirtual

Stop playback and rewind to beginning.

Previous state must be Running or Open. New state will be Open.

Reimplemented from Stream.

◆ singleStep()

bool singleStep ( )

Play exactly one frame at current position and then pause.

Previous state must be Open. New state will be Open.

◆ replayFrame()

bool replayFrame ( )

Play frame at current position again and then pause.

Previous state must be Open. New state will be Open.

◆ depthK()

bool depthK ( mat3 & K ) const

overridevirtual

Get depth camera intrinsics.

Reimplemented from RGBDStream.

◆ colorK()

bool colorK ( mat3 & K ) const

overridevirtual

Get color camera intrinsics.

Reimplemented from RGBDStream.

◆ depthToColorT()

bool depthToColorT ( mat4 & T ) const

overridevirtual

Get transformation from depth to color camera.

Reimplemented from RGBDStream.

◆ matrix()

mat4 matrix ( ) const

overridevirtual

Get current transformation matrix.

Reimplemented from Data.

◆ setMatrix()

void setMatrix ( const mat4 & m )

overridevirtual

Set current transformation matrix.

Reimplemented from Data.

◆ supportsColor()

bool supportsColor ( ) const

inlineoverridevirtual

Returns true if color images are supported.

Implements RGBDStream.

◆ uuid()

std::string uuid ( )

overridevirtual

Get the UUID of the playback stream.

Implements Stream.

◆ depthImageWidth()

int depthImageWidth ( ) const

overridevirtual

Get depth image width.

Implements RGBDStream.

◆ depthImageHeight()

int depthImageHeight ( ) const

overridevirtual

Get depth image height.

Implements RGBDStream.

◆ colorImageWidth()

int colorImageWidth ( ) const

overridevirtual

Get color image width.

Implements RGBDStream.

◆ colorImageHeight()

int colorImageHeight ( ) const

overridevirtual

Get color image height.

Implements RGBDStream.

◆ describe()

std::string describe ( ) const

overridevirtual

Get a human-readable description of the playback stream.

Reimplemented from ImageStream.

◆ setSelection()

void setSelection ( const Selection & sel )

overridevirtual

Set the selection of frames/sensors for playback.

Reimplemented from Selectable.

◆ setFocus()

void setFocus ( int index )

overridevirtual

Set the focus to a specific sensor.

Reimplemented from Selectable.

◆ size()

int size ( ) const

overridevirtual

Get the number of sensors/sequences.

Reimplemented from ImageStream.

◆ serialNumber()

virtual std::string serialNumber ( ) const

overridevirtual

Get the serial number of the device.

Reimplemented from RGBDStream.

◆ onRGBDDataComponentChanged()

void onRGBDDataComponentChanged ( )

protectedvirtual

Called when the RGBDDataComponent changes.

Reimplemented from RGBDStream.

◆ openImpl()

bool openImpl ( )

overrideprotectedvirtual

Load recording. Returns false when loading failed.

Implements Stream.

◆ closeImpl()

bool closeImpl ( )

overrideprotectedvirtual

Close stream.

Returns whether the operation was successful. This method should be used to free any resources that were allocated during the openImpl() call and to disconnect from any devices. Implementers must ensure that all resources are properly released to avoid memory leaks.

Implements Stream.

◆ startImpl()

bool startImpl ( )

overrideprotectedvirtual

Start playback of loaded file.

Previous state must be Stopped or Paused. New state will be Playing.

Implements Stream.

◆ stopImpl()

bool stopImpl ( )

overrideprotectedvirtual

Pause playback at current position.

Previous state must be Playing. New state will be Paused.

Implements Stream.

◆ doWork()

std::optional< WorkContinuation > doWork ( )

overrideprotectedvirtual

Perform playback work (internal).

Implements Stream.

The documentation for this class was generated from the following file:

RGBD/Include/ImFusion/RGBD/RGBDPlayback.h

Detailed Description

Public Types

Public Member Functions

Public Attributes

Protected Member Functions

Additional Inherited Members

Member Enumeration Documentation

◆ LoopingMode

Member Function Documentation

◆ reset()

◆ singleStep()

◆ replayFrame()

◆ depthK()

◆ colorK()

◆ depthToColorT()

◆ matrix()

◆ setMatrix()

◆ supportsColor()

◆ uuid()

◆ depthImageWidth()

◆ depthImageHeight()

◆ colorImageWidth()

◆ colorImageHeight()

◆ describe()

◆ setSelection()

◆ setFocus()

◆ size()

◆ serialNumber()

◆ onRGBDDataComponentChanged()

◆ openImpl()

◆ closeImpl()

◆ startImpl()

◆ stopImpl()

◆ doWork()