Abstract:Facial analysis is a key component in a wide range of applications such as security, autonomous driving, entertainment, and healthcare. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the transfer of thermal style to RGB faces. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the T-FAKE dataset, a large-scale synthetic thermal dataset of faces. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Our code and models are available at <a class="link-external link-https" href="https://github.com/phflot/tfake" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of insufficient data in the field of thermal facial landmark detection. Specifically, although facial datasets of RGB images are already very rich, the importance of thermal imaging in life sciences, medicine, and biometrics is increasing day by day, while the corresponding annotated thermal imaging facial data is relatively scarce. This has restricted the development of thermal facial landmark detection methods. To solve this problem, the authors introduced the **T - FAKE dataset**, which is a large - scale synthetic thermal imaging facial dataset containing sparse and dense facial landmark annotations. Through this dataset, they hope to improve the effect of thermal facial landmark detection and promote the development of related applications. In addition, they also proposed a new RGB2Thermal loss function for converting RGB images into thermal imaging images, thereby generating realistic synthetic thermal imaging data. ### Main contributions 1. **T - FAKE dataset**: This is the first large - scale synthetic thermal imaging facial dataset, containing sparse (70 points) and dense (478 points) facial landmark annotations. 2. **RGB2Thermal loss function**: A new loss function used to overcome the limitations of training data under laboratory conditions and improve the generalization ability for images in the wild environment. 3. **Model training**: For the first time, a dense thermal imaging facial landmark detector was trained, and combined with a multi - modal RGB + thermal imaging sparse landmark detector, highly structured benchmark tests were carried out on different landmark specifications and modalities. ### Key technologies of the solution - **RGB2Thermal loss function**: This loss function includes three key parts: - **Supervised data term**: Controls the generation of thermal imaging faces based on a small number of RGB - thermal imaging paired samples. - **Wasserstein distance term**: Aligns the patch distributions of the generated synthetic thermal imaging images and the real thermal imaging images. - **Clinical temperature statistical prior information term**: Adjusts according to the clinical temperature statistical information of different facial regions. Through these technologies, the authors not only generated a high - quality synthetic thermal imaging dataset but also verified the effectiveness of their method on multiple benchmark datasets, showing significant performance improvements. ### Summary This paper fills the data gap in the field of thermal facial landmark detection and promotes the development of related technologies by creating a large - scale synthetic thermal imaging facial dataset and proposing a new RGB2Thermal loss function.

T-FAKE: Synthesizing Thermal Images for Facial Landmarking

Modality-transfer Generative Adversarial Network and Dual-Level Unified Latent Representation for Visible Thermal Person Re-Identification

A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset

A Novel Fully Annotated Thermal Infrared Face Dataset: Recorded in Various Environment Conditions and Distances From The Camera

Thermal Image Generation for Robust Face Recognition

Generating Thermal Human Faces for Physiological Assessment using Thermal Sensor Auxiliary Labels

A Synthesis-Based Approach for Thermal-to-Visible Face Verification

CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark

Generating Thermal Image Data Samples using 3D Facial Modelling Techniques and Deep Learning Methodologies

Multi-spectral Facial Landmark Detection

Enhancing thermal facial recognition leveraging large datasets and hybrid algorithms

Thermal to Visible Synthesis of Face Images using Multiple Regions

Facial Expression Recognition Enhanced by Thermal Images Through Adversarial Learning.

Vesselness features and the inverse compositional AAM for robust face recognition using thermal IR

Influence of Thermal Imagery Resolution on Accuracy of Deep Learning based Face Recognition

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Precise Facial Landmark Detection by Reference Heatmap Transformer

Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum

3D face reconstruction with dense landmarks

ThermoNeRF: Joint RGB and Thermal Novel View Synthesis for Building Facades using Multimodal Neural Radiance Fields