Abstract:Fiducial markers have been broadly used to identify objects or embed messages that can be detected by a camera. Primarily, existing detection methods assume that markers are printed on ideally planar surfaces. Markers often fail to be recognized due to various imaging artifacts of optical/perspective distortion and motion blur. To overcome these limitations, we propose a novel deformable fiducial marker system that consists of three main parts: First, a fiducial marker generator creates a set of free-form color patterns to encode significantly large-scale information in unique visual codes. Second, a differentiable image simulator creates a training dataset of photorealistic scene images with the deformed markers, being rendered during optimization in a differentiable manner. The rendered images include realistic shading with specular reflection, optical distortion, defocus and motion blur, color alteration, imaging noise, and shape deformation of markers. Lastly, a trained marker detector seeks the regions of interest and recognizes multiple marker patterns simultaneously via inverse deformation transformation. The deformable marker creator and detector networks are jointly optimized via the differentiable photorealistic renderer in an end-to-end manner, allowing us to robustly recognize a wide range of deformable markers with high accuracy. Our deformable marker system is capable of decoding 36-bit messages successfully at ~29 fps with severe shape deformation. Results validate that our system significantly outperforms the traditional and data-driven marker methods. Our learning-based marker system opens up new interesting applications of fiducial markers, including cost-effective motion capture of the human body, active 3D scanning using our fiducial markers' array as structured light patterns, and robust augmented reality rendering of virtual objects on dynamic surfaces.

Generation of fiducial marker dictionaries using Mixed Integer Linear Programming

Optimal Fitting Strategy for Modulated Illumination Localization Microscopy

Large-Scale Indoor Camera Positioning Using Fiducial Markers

Simultaneous Multi-View Camera Pose Estimation and Object Tracking with Square Planar Markers

DeepTag: A General Framework for Fiducial Marker Design and Detection

DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers

An Automatic Registration Approach to Laser Point Sets Based on Multidiscriminant Parameter Extraction

Detection of Binary Square Fiducial Markers Using an Event Camera

Tape-Shaped, Multiscale, and Continuous-Readable Fiducial Marker for Indoor Navigation and Localization Systems

Event-Based high-speed low-latency fiducial marker tracking

TopoTag: A Robust and Scalable Topological Fiducial Marker System

Efficient and Outlier-Robust Simultaneous Pose and Correspondence Determination by Branch-and-Bound and Transformation Decomposition

HydraMarker: Efficient, Flexible, and Multifold Marker Field Generation

Certifiably Optimal Mutual Localization With Anonymous Bearing Measurements

A Machine Learning Approach to Robot Localization Using Fiducial Markers in RobotAtFactory 4.0 Competition

Automatic Detection of Fiducial Marker Center Based on Shape Index and Curvedness

SIGMA: Scale-Invariant Global Sparse Shape Matching

Instance by Instance: An Iterative Framework for Multi-instance 3D Registration

Marker-based Visual SLAM leveraging Hierarchical Representations

Non-convex Pose Graph Optimization in SLAM via Proximal Linearized Riemannian ADMM

Optimal Camera Planning Under Versatile User Constraints in Multi-Camera Image Processing Systems