Marker Detection

Comprehensive guide to marker detection.

Collaboration diagram for Marker Detection:

Comprehensive guide to marker detection.

This page provides detailed information and code examples for detecting various types of markers using the MarkerDetection and MarkerConfiguration classes. These tools are essential for computer vision applications such as camera calibration, pose estimation, and augmented reality.

Marker Detection Overview

Marker detection is a fundamental computer vision technique used for camera calibration, pose estimation, and augmented reality applications. The MarkerDetection class provides a unified interface for detecting various types of markers, while the MarkerConfiguration class allows you to configure the specific marker type and parameters.

Applications

• Camera Calibration: Determine intrinsic camera parameters (focal length, principal point, distortion coefficients)
• Pose Estimation: Calculate the 3D position and orientation of cameras or objects
• Augmented Reality: Track real-world objects to overlay virtual content
• Robotics: Localize robots or tools in 3D space
• Medical Imaging: Track surgical instruments or patient anatomy
• Quality Control: Measure object positions and orientations in manufacturing

Typical Workflow

1. Configure Marker Type: Set up the appropriate marker configuration
2. Detect Markers: Process images to find marker corners
3. Extract Points: Get 2D image points and corresponding 3D object points
4. Estimate Pose: Calculate camera pose using camera parameters
5. Validate Results: Check detection quality and pose accuracy

Available Marker Types

The ImFusion SDK supports several marker types, each with different characteristics and use cases:

Chessboard

• Description: Traditional black and white checkerboard pattern
• Pros: Simple, reliable, widely supported
• Best for: Camera calibration
• Configuration: Grid size, cell size, detector parameters

Charuco Board

• Description: Chessboard with embedded ArUco markers
• Pros: Robust detection, handles partial occlusion, good for calibration
• Best for: Camera calibration
• Configuration: Grid size, cell size, marker size, dictionary, detector parameters

ArUco Markers

• Description: Binary-coded square markers with unique IDs
• Pros: Fast detection, unique identification, handles occlusion
• Best for: Pose estimation, augmented reality, tracking
• Configuration: Dictionary, marker size, detector parameters

AprilTag

• Description: Robust fiducial markers with error correction
• Pros: Very robust, handles lighting variations, unique IDs
• Cons: Larger markers needed for good detection
• Best for: Industrial applications, outdoor use, long-range detection
• Configuration: Family, marker size, detector parameters

STag

• Description: Circular fiducial markers with high accuracy
• Pros: Sub-pixel accuracy, handles perspective distortion well
• Cons: Limited number of unique markers
• Best for: High-precision applications
• Configuration: Diameter, detector parameters

Circle Board

• Description: Pattern of circular markers arranged in a grid
• Pros: Rotation invariant, handles perspective well
• Cons: Requires good contrast, can be affected by lighting
• Best for: Camera calibration, industrial applications
• Configuration: Grid size, circle spacing, detector parameters

Basic Marker Detection Usage

Basic Chessboard Detection

The following example demonstrates basic chessboard detection:

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/MarkerDetection.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for chessboard
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::Chessboard);
 
// Configure chessboard parameters
MarkerConfiguration::ChessboardInfo chessboardParams;
chessboardParams.gridSize = vec2i(9, 7);  // 9x7 internal corners
chessboardParams.cellSize = vec2(30, 30); // 30 mm cell size
config.setChessboardParameters(chessboardParams);
 
// Create marker detector
MarkerDetection detector;
detector.setMarkerConfiguration(config);
 
// Load and process image
std::unique_ptr<SharedImage> image = loadImage("chessboard.jpg");
bool success = detector.detectMarker(image.get());
 
if (success) {
    // Extract detected points
    std::vector<vec3> objectPoints;
    std::vector<vec2> imagePoints;
    detector.detections(objectPoints, imagePoints);
    
    LOG_INFO("Detected " << imagePoints.size() << " corners");
    
    // Use points for camera calibration or pose estimation
    // ...
}

Charuco Board Detection

Charuco boards combine the benefits of chessboards and ArUco markers:

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/MarkerDetection.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for Charuco board
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::CharucoBoard);
 
// Configure Charuco board parameters
MarkerConfiguration::CharucoBoardInfo charucoParams;
charucoParams.gridSize = vec2i(9, 7);     // 9x7 grid
charucoParams.cellSize = 30.0;            // 30 mm cell size
charucoParams.markerSize = 10.0;          // 10 mm marker size
charucoParams.dictionary = 0;             // ArUco dictionary
charucoParams.minAdjacentMarkers = 2;     // Minimum adjacent markers for corner detection
config.setCharucoBoardParameters(charucoParams);
 
// Create marker detector
MarkerDetection detector;
detector.setMarkerConfiguration(config);
 
// Enable automatic size hint detection for better performance
detector.setDetectSizeHintAutomatically(true);
 
// Process image
std::unique_ptr<SharedImage> image = loadImage("charuco_board.jpg");
bool success = detector.detectMarker(image.get());
 
if (success) {
    // Extract detected points and marker information
    std::vector<vec3> objectPoints;
    std::vector<vec2> imagePoints;
    std::vector<int> pointIds;
    std::vector<std::vector<vec2>> markerCorners;
    std::vector<int> markerIds;
    
    detector.detections(objectPoints, imagePoints, &pointIds, &markerCorners, &markerIds);
    
    LOG_INFO("Detected " << imagePoints.size() << " corners");
    LOG_INFO("Detected " << markerIds.size() << " markers");
    
    // Use for camera calibration or pose estimation
    // ...
}

ArUco Marker Detection

ArUco markers are ideal for pose estimation and tracking:

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/MarkerDetection.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for ArUco markers
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::ArucoMarker);
 
// Configure ArUco marker parameters
MarkerConfiguration::ArucoMarkerInfo arucoParams;
arucoParams.id = 0;                       // Marker ID to detect
arucoParams.dictionary = 3;               // ArUco dictionary
arucoParams.size = 50.0;                  // 50 mm marker size
arucoParams.T = mat4::Identity();         // Marker pose in world coordinates
config.addArucoMarker(arucoParams);
 
// Add more markers if needed
MarkerConfiguration::ArucoMarkerInfo arucoParams2;
arucoParams2.id = 1;
arucoParams2.dictionary = 3;
arucoParams2.size = 50.0;
arucoParams2.T = mat4::Identity();
arucoParams2.T.block<3, 1>(0, 3) = vec3(100.0, 0.0, 0.0); // Translate 100 mm in x direction
config.addArucoMarker(arucoParams2);
 
// Create marker detector
MarkerDetection detector;
detector.setMarkerConfiguration(config);
 
// Configure detector parameters for better performance
MarkerConfiguration::DetectorParameters detectorParams;
detectorParams.adaptiveThreshWinSizeMin = 3;
detectorParams.adaptiveThreshWinSizeMax = 23;
detectorParams.minMarkerPerimeterRate = 0.03;
detectorParams.maxMarkerPerimeterRate = 4.0;
detectorParams.cornerRefinementMethod = MarkerConfiguration::DetectorParameters::Subpix;
detectorParams.cornerRefinementWinSize = 5;
 
// Process image
std::unique_ptr<SharedImage> image = loadImage("aruco_markers.jpg");
bool success = detector.detectMarker(image.get());
 
if (success) {
    // Extract detected markers
    std::vector<vec3> objectPoints;
    std::vector<vec2> imagePoints;
    std::vector<int> pointIds;
    std::vector<std::vector<vec2>> markerCorners;
    std::vector<int> markerIds;
    
    detector.detections(objectPoints, imagePoints, &pointIds, &markerCorners, &markerIds);
    
    LOG_INFO("Detected " << markerIds.size() << " markers");
    
    // Process each detected marker
    for (size_t i = 0; i < markerIds.size(); ++i) {
        LOG_INFO("Marker ID: " << markerIds[i]);
        // Use marker corners for pose estimation
        // ...
    }
}

Pose Estimation

Once markers are detected, you can estimate the camera pose:

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/MarkerDetection.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Set up marker detection (using previous examples)
MarkerConfiguration config;
// ... configure marker type and parameters ...
 
MarkerDetection detector;
detector.setMarkerConfiguration(config);
 
// Set camera parameters (intrinsic matrix and distortion coefficients)
mat3 K;  // Camera intrinsic matrix
vec5 dist;  // Distortion coefficients
// ... load or calculate camera parameters ...
 
detector.setCameraParameters(K, dist);
 
// Process image
std::unique_ptr<SharedImage> image = loadImage("markers.jpg");
bool success = detector.detectMarker(image.get());
 
if (success) {
    // Estimate camera pose
    double reprojectionError;
    std::vector<vec2> reprojectedPoints;
    mat4 cameraPose = detector.pose(&reprojectionError, &reprojectedPoints);
    
    if (cameraPose != mat4::Zero()) {
        LOG_INFO("Camera pose estimated successfully");
        LOG_INFO("Reprojection error: " << reprojectionError << " pixels");
        
        // Extract rotation and translation
        mat3 rotation = cameraPose.block<3, 3>(0, 0);
        vec3 translation = cameraPose.block<3, 1>(0, 3);
        
        LOG_INFO("Translation: [" << translation.x() << ", " << translation.y() << ", " << translation.z() << "]");
        
        // Use pose for augmented reality, robotics, etc.
        // ...
    } else {
        LOG_WARN("Pose estimation failed");
    }
}

Marker Generation

The MarkerGenerationAlgorithm class allows you to generate various types of markers for printing and use in calibration or tracking applications.

Basic Marker Generation

Generate a chessboard pattern for camera calibration:

#include <ImFusion/Vision/MarkerGenerationAlgorithm.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for chessboard
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::Chessboard);
 
// Configure chessboard parameters
MarkerConfiguration::ChessboardInfo chessboardParams;
chessboardParams.gridSize = vec2i(9, 7);  // 9x7 internal corners
chessboardParams.cellSize = vec2(30, 30); // 30 mm cell size
config.setChessboardParameters(chessboardParams);
 
// Create marker generation algorithm
MarkerGenerationAlgorithm generator;
 
// Set marker configuration
generator.setMarkerConfig(config);
 
// Configure generation parameters
generator.p_padding = 50;        // 50 pixels padding around the marker
generator.p_dpi = 300;           // 300 DPI for high-quality printing
generator.p_outputSVGPath = "chessboard.svg"; // Save as SVG file
 
// Generate the marker
generator.compute();

Charuco Board Generation

Generate a Charuco board for robust calibration:

#include <ImFusion/Vision/MarkerGenerationAlgorithm.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for Charuco board
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::CharucoBoard);
 
// Configure Charuco board parameters
MarkerConfiguration::CharucoBoardInfo charucoParams;
charucoParams.gridSize = vec2i(9, 7);     // 9x7 grid
charucoParams.cellSize = 30.0;            // 30 mm cell size
charucoParams.markerSize = 10.0;          // 10 mm marker size
charucoParams.dictionary = 0;             // ArUco dictionary
charucoParams.minAdjacentMarkers = 2;     // Minimum adjacent markers for corner detection
config.setCharucoBoardParameters(charucoParams);
 
// Create marker generation algorithm
MarkerGenerationAlgorithm generator;
generator.setMarkerConfig(config);
 
// Configure generation parameters
generator.p_padding = 100;       // 100 pixels padding
generator.p_dpi = 600;           // 600 DPI for very high quality
generator.p_outputSVGPath = "charuco_board.svg";
 
// Generate the marker
generator.compute();

Circle Board Generation

Generate a circle board pattern:

#include <ImFusion/Vision/MarkerGenerationAlgorithm.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Create marker configuration for circle board
MarkerConfiguration config;
config.setMarkerType(MarkerConfiguration::CircleBoard);
 
// Configure circle board parameters
MarkerConfiguration::CircleBoardInfo circleParams;
circleParams.gridSize = vec2i(9, 7);      // 9x7 grid
circleParams.circleSpacing = 20.0;        // 20 mm spacing between circles
circleParams.symmetric = true;            // Symmetric pattern
config.setCircleBoardParameters(circleParams);
 
// Create marker generation algorithm
MarkerGenerationAlgorithm generator;
generator.setMarkerConfig(config);
 
// Configure generation parameters
generator.p_padding = 75;        // 75 pixels padding
generator.p_dpi = 300;           // 300 DPI
generator.p_circleDiameter = 8.0; // 8 mm circle diameter
generator.p_outputSVGPath = "circle_board.svg";
 
// Generate the marker
generator.compute();

Generation Parameters

The marker generation algorithm provides several parameters to control the output:

DPI and Resolution

• DPI: Controls the resolution of the generated marker (typically 300-600 DPI for printing)
• Padding: Adds space around the marker for easier handling and printing
• Circle Diameter: For circle boards, controls the size of individual circles

Output Formats

• SVG: Vector format for scalable printing and editing
• Direct Image: Access the generated image programmatically for further processing via takeOutput

Visualization and Debugging

Visualization Example

Create detection visualization for debugging and validation:

#include <ImFusion/Base/SharedImage.h>
#include <ImFusion/Vision/MarkerDetection.h>
#include <ImFusion/Vision/MarkerConfiguration.h>
 
using namespace ImFusion;
 
// Set up marker detection
MarkerConfiguration config;
// ... configure marker type and parameters ...
 
MarkerDetection detector;
detector.setMarkerConfiguration(config);
 
// Load input image
std::unique_ptr<SharedImage> inputImage = loadImage("markers.jpg");
 
// Create output image for visualization
auto detectionImage = inputImage->clone();
 
// Perform detection with visualization
bool success = detector.detectMarker(inputImage.get(), detectionImage.get());
 
if (success) {
    // Save visualization
    saveImage(detectionImage.get(), "detection_result.jpg");
    
    // Extract and log detection results
    std::vector<vec3> objectPoints;
    std::vector<vec2> imagePoints;
    std::vector<int> pointIds;
    std::vector<std::vector<vec2>> markerCorners;
    std::vector<int> markerIds;
    
    detector.detections(objectPoints, imagePoints, &pointIds, &markerCorners, &markerIds);
    
    LOG_INFO("Detection Results:");
    LOG_INFO("  Total corners detected: " << imagePoints.size());
    LOG_INFO("  Total markers detected: " << markerIds.size());
    
    // Log individual marker information
    for (size_t i = 0; i < markerIds.size(); ++i) {
        LOG_INFO("  Marker " << i << ": ID=" << markerIds[i] << ", Corners=" << markerCorners[i].size());
    }
    
    // Log corner information
    for (size_t i = 0; i < imagePoints.size(); ++i) {
        LOG_INFO("  Corner " << i << ": (" << imagePoints[i].x() << ", " << imagePoints[i].y() << ")");
        if (!pointIds.empty()) {
            LOG_INFO("    Point ID: " << pointIds[i]);
        }
    }
}

Best Practices

Marker Type Selection Guidelines

Choose the appropriate marker type based on your application:

Camera Calibration

• Chessboard: Simple, reliable, good for static environments
• Charuco Board: Robust, handles partial occlusion, good for dynamic environments
• Circle Board: Rotation invariant, good for industrial applications

Pose Tracking

• ArUco Markers: Fast detection, unique IDs, good for real-time applications
• AprilTag: Very robust, handles lighting variations, good for outdoor use
• STag: High precision, good for medical or precision applications

Parameter Tuning Guidelines

Size and Scale Parameters

• Marker Size: Should be large enough for reliable detection (typically 20-100 mm)
• Grid Size: Balance between detection robustness and computational cost
• Cell Size: Should match the physical size of your calibration target

Detection Parameters

• Adaptive Threshold: Adjust for different lighting conditions
• Corner Refinement: Use subpixel refinement for higher accuracy
• Minimum Perimeter Rate: Adjust based on marker distance from camera

Environmental Considerations

Lighting

• Even Illumination: Avoid shadows and specular reflections
• Contrast: Ensure good contrast between marker and background
• Dynamic Range: Consider using HDR imaging for challenging lighting

Motion and Blur

• Exposure Time: Use short exposure to minimize motion blur
• Frame Rate: Higher frame rates for moving targets
• Stabilization: Consider image stabilization for handheld cameras

Validation and Quality Control

Reprojection Error

• Acceptable Range: Typically < 1-2 pixels for good calibration
• Monitoring: Track reprojection error over time
• Thresholds: Set appropriate thresholds for your application

Coverage and Distribution

• Image Coverage: Ensure markers cover the full image area
• Depth Variation: Include markers at different distances
• Orientation: Vary marker orientations for robust calibration

Troubleshooting

Common Issues and Solutions

No Detection

• Symptoms: No markers detected despite being visible
• Solutions:
  • Check marker configuration parameters
  • Verify marker size and grid size
  • Adjust detector parameters for lighting conditions
  • Ensure good contrast between marker and background

Poor Detection Accuracy

• Symptoms: High reprojection error, inconsistent results
• Solutions:
  • Use subpixel corner refinement
  • Increase marker size or improve image quality
  • Check for motion blur or camera shake
  • Verify camera parameters are correct

Slow Performance

• Symptoms: Long detection times
• Solutions:
  • Use size hints to limit search area
  • Reduce grid size or number of markers
  • Optimize detector parameters
  • Consider using GPU acceleration where available

False Positives

• Symptoms: Detecting markers where none exist
• Solutions:
  • Adjust minimum perimeter rate
  • Increase minimum corner distance
  • Use more restrictive detector parameters
  • Enable duplicate marker removal

See also

MarkerDetection class, MarkerConfiguration class