Resource system to retrieve at runtime binary data embedded into the application/library. More...

Collaboration diagram for Resource System:

Detailed Description

Resource system to retrieve at runtime binary data embedded into the application/library.

The ImFusion Resource system is a light-weight platform-independent mechanism to embed binary data in a library/executable. It is somewhat similar to the Qt Resource System, however does not have any third-party dependencies except for ImFusionCore. It consists of two layers/steps: The embedding mechanism of the data at compilation time, and the retrieval of the data at runtime.

Compiling Resources into the Application

The ImFusionResourceCompiler

Use the ImFusionResourceCompiler executable to generate a C++ file to build and link into your library/application. The resulting file will contain the specified input files as well as the code needed to implement a Repository and automatically register it with the ImFusion::Resource system (optional). Storing files as external zip-archives is also supported and should be preferred for large files (>100MB), since embedding large files considerably slows down compilation.

ImFusionResourceCompiler [options] <list of input files>:
 
Mandatory arguments:
  -n [ --repo-name ] arg     Repository Name
 
Optional arguments:
  -o [ --output-file ] arg   Output file, will output to stdout if not provided
  -d [ --base-dir ] arg      Path the embedded resources shall be relative to,
                             defaults to the current working dir if not provided
  --chunk-size arg           Chunk size in bytes into which larger resources
                             are split up (default: 2MB). Ignored if `--external-storage` is passed.
  --help                     display this help message
  --compress                 Compress the input file before embedding
  --encrypt                  Encrypt the input file before embedding (mutually exclusive with `--encrypt-asym`)
  --external-storage arg     Store the input files as an external .zip archive in the given directory. The directory must exist.
  --generate-key-file arg    Generates a private key for asymmetric encryption and stores it in a file at the given path. 
                             The file is created if needed and it is overwritten if it exists.
  --encrypt-asym arg         Encrypts and signs the resources using asymmetric encryption (Ed25519). 
                             A valid path to the key file (generated with `--generate-key-file`) must be provided (mutually exclusive with `--encrypt`)
  --strip-c-style-comments   Strip C-style comments from the file contents
  --no-auto-register         Do not automatically register the repository on
                             library load. Instead provide a header file so
                             that developers can register the repo on demand.

A minimal example to embed two files could look like this:

ImFusionResourceCompiler --repo-name MyRepo --output-file MyRepo.cpp --base-dir data data/mesh.obj data/texture.png

This will create the file MyRepo.cpp containing code that registers a Repository with name MyRepo. This repository then provides the contents of the files data/mesh.obj and data/texture.png under the names mesh.obj and texture.png. Note that you need to include this file into your build system so that it gets compiled and linked into the target library/executable. The target library/executable needs to link against ImFusionCore.

CMake Infrastructure

The ImFusion CMake infrastructure provides a convenience function to perform the above steps automatically with a single function call.

imfusion_compile_resource_repository(<RepositoryName>
    FILES <List of Files to Embed>
    [ TARGET <Target Name> ]
    [ BASE_DIR <RelativePath> ]
    [ WORKING_DIR <WorkingDirectory> ]
    [ OUTPUT_FILE <Output File Name> ]
    [ CHUNK_SIZE <Optional Chunk Size in Bytes> ]
    [ EXTERNAL_STORAGE <Directory where external resources will be stored>]
    [ ENCRYPT ]
    [ ENCRYPT_ASYM <Path to the generated key-file. Use the --generate-key-file (--g) option of ResourceCompiler to generate this file>]
    [ COMPRESS ]
    [ STRIP_C_STYLE_COMMENTS ]
    [ NO_AUTO_REGISTER ]
)

For instance, all you need to do to create the resource repository from the above example for your library MyLibrary is the following:

# define and configure library
add_library(MyLibrary)
target_link_libraries(MyLibrary PRIVATE ImFusion::ImFusionCore)
 
# define and create resource repository
imfusion_compile_resource_repository(MyRepo
    BASE_DIR data/
    FILES data/mesh.obj data/texture.png
)

ExternalStorage

The option EXTERNAL_STORAGE allows to store files in an external zip-archive and avoid embedding them as binary. This is particularly beneficial if you plan to embed large resources (>100 MB), which would slow down/halt the compilation. You can provide a custom directory as argument to this option. Your resources will then be stored there. When querying an externally stored resource, the following directories are searched (in order) for the corresponding zip archive:

Standard, platform-specific path relative to the ImFusionCore library (normally used by internal ImFusion resources)
Subfolder /Resources of the executable's directory
Custom folder specified by the environment variable IMFUSION_RESOURCES_DIR(mainly for development)

The third option, using the IMFUSION_RESOURCES_DIR environment variable, is mainly useful during development. If you use this option in production, you will need to ensure that IMFUSION_RESOURCES_DIR is configured correctly on all systems running your application. In fact, if a custom resources folder is specified, resources are not automatically relocated during installation. We recommend installing the resources in a /Resources subfolder of the executable's directory, so that they are found at runtime (see above for a list of search-paths at runtime).

Resources that are stored in external archives via EXTERNAL_STORAGE can be encrypted with standard symmetric encryption (ENCRYPT) or with asymmetric encryption (ENCRYPT_ASYM). The latter allows different resource-archives that have been signed with the same key to be interchangeable. Note that this operation does however increase the time required to create such archives. To use this option, a valid key-file must be first generated using directly the --generate-key-file (--g) of the ResourceCompiler executable. Once the key is generated, it must be safely store for later re-use, and can be passed as argument to ENCRYPT_ASYM. If you wish to later exchange your deployed resource-archive with a new one, just run this function again with the same key-file and the new resource-files. Since the key used is the same, the old archive can be safely replaced with the new one and the Resource System will be able to load it.

Note: The repository output file name must be unique. If unspecified it will use ${CMAKE_CURRENT_BINARY_DIR}/${RepositoryName}.cpp as default output file name. Thus, if you want to have multiple repositories of the same name in a single target you will need to specify individual output file names for each of them.

Retrieving Compiled Resources at Runtime

Use the functions in the ImFusion::Resource namespace to retrieve compiled resources at runtime. Resource::query() will search all registered repositories for a resource with the given name and return the first one it finds (the search order is undefined). You can provide an optional repository name to filter and avoid ambiguities.

std::optional<ByteBuffer> mesh = Resource::query("MyRepo", "mesh.obj");
if (mesh)
{
    processMesh(*mesh);
}

Registration of Resource Repositories

A Resource::Repository must be registered with the resource system before it can be used. Manual registration of custom repositories is done using the Resource::addRepository() function.

Repositories created with the ImFusionResourceCompiler as described above are registered automatically by default using the static initialization pattern. Therefore, you normally do not need to do anything else in addition to define the resource repository via CMake.

You can opt-out of this feature by passing the no-auto-register flag. In this case the resource compiler will output a header file next to the cpp file providing the declaration of the created repository. You can then include this header file, instantiate an instance of the repository, and register it with the resource system as needed.

The FilesystemRepository

During development it can be convenient to not always embed all resources on every compilation but instead load them dynamically from the filesystem at runtime. The FilesystemRepository enables you to do this while maintaining the same retrieval API through Resource::query(). For instance, you could setup your build system to use the ImFusionResourceCompiler only for production builds and setup a FilesystemRepository with the exact same semantics for development builds.

Namespaces
namespace	ImFusion::Resource
	ImFusion Resource System to store binary data in the executable/library and retrieve it at runtime.

Classes
class	ImFusion::Resource::FilesystemRepository
	Repository for loading resources from the local filesystem. More...
class	ImFusion::Resource::Repository
	Base interface for a ImFusion resource repository. More...

Functions
void	ImFusion::Resource::addRepository (const Repository *rep)
	Add the given Repository to the set of registered repositories.
void	ImFusion::Resource::removeRepository (const Repository *rep)
	Remove the given Repository from the set of registered repositories.
const Repository *	ImFusion::Resource::locate (const std::string_view &repositoryName, const std::string_view &resourceName)
	Searches the registered repositories if they provide a resource identified by the given name.
std::optional< ByteBuffer >	ImFusion::Resource::query (const std::string_view &repositoryName, const std::string_view &resourceName)
	Queries the registered repositories for the resource identified by the given name.

Function Documentation

◆ locate()

const Repository * ImFusion::Resource::locate	(	const std::string_view &	repositoryName,
		const std::string_view &	resourceName )

#include <ImFusion/Core/Resource/Resource.h>

Searches the registered repositories if they provide a resource identified by the given name.

You can use this function to check if a resource is available without actually retrieving it. If there are multiple repositories matching the given query it is undefined which one is returned.

Parameters

repositoryName	Optional filter to restrict the search to repositories of the given name. Will search all registered repositories if empty.
resourceName	Name of the resource to search for

Returns: A registered Repository so that repository->query(resourceName) returns a valid ByteBuffer, nullptr otherwise.

◆ query()

std::optional< ByteBuffer > ImFusion::Resource::query	(	const std::string_view &	repositoryName,
		const std::string_view &	resourceName )