Abstract:Tactile sensing has recently been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration information from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation. Through the frequency response of the vibration induced by the material and thermal features, like an estimate of the thermal power loss of the finger, we show that it is possible to identify materials in less than half a second. Moreover, a comparison between the use of vibration only and multi-modal identification shows that both recognition time and classification errors are reduced by adding thermal information.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve two main problems in material identification through tactile sensors: 1. **Limitations of single - modal information**: Most existing material identification methods rely solely on vibration signals. Although these methods can achieve good classification results in some cases, they ignore other important perceptual information (such as temperature). This may lead to insufficient identification accuracy and speed. 2. **Excessive identification time**: Traditional material identification methods based on vibration information usually require at least one second to collect enough data to complete the classification. This is a significant bottleneck for real - time applications. To solve these problems, the author proposes a multimodal (vibration and heat) material identification method based on recursive Bayesian estimation. Specifically, this method combines vibration and heat features to achieve rapid and accurate identification of materials in less than 0.5 seconds. In addition, the experimental results show that adding thermal information not only shortens the identification time but also reduces the number of classification errors. ### Method overview 1. **Data collection and pre - processing**: - Use the SynTouch BioTAC fingertip sensor, which provides multiple sensing modes, including pressure, vibration, temperature, heat flux, and skin deformation. - Perform low - pass filtering and band - pass filtering on the vibration signal to generate vibration features. - Calculate thermal features such as heat flux and temperature change rate. 2. **Feature extraction**: - Vibration features: Use the fast Fourier transform (FFT) to convert the vibration signal to the frequency domain and reduce the dimension through principal component analysis (PCA). - Thermal features: Calculate the average value, slope, and regression error of the heat flux to form a three - dimensional thermal feature vector. 3. **Recursive Bayesian estimation**: - Establish a Gaussian mixture model (GMM) to represent the probability distribution of vibration and thermal features of each material. - Use the recursive Bayesian formula to update the posterior probability of each material and gradually approach the correct material. ### Experimental results - **Vibration information only**: The average identification time is 0.36 seconds, the standard deviation is 0.2 seconds, and the misclassification rate is 1.21%. - **Multimodal information (vibration + heat)**: The average identification time is 0.28 seconds, the standard deviation is 0.09 seconds, and the misclassification rate is 1.02%. Compared with using only vibration information, both the identification time and the error rate have been significantly improved. Through this method, researchers have not only improved the speed and accuracy of material identification but also demonstrated the potential of multimodal perception in robotic touch. ### Conclusions and future work The multimodal recursive Bayesian estimation method proposed in this paper performs excellently in material identification and can achieve high - precision classification in a very short time. Future work can further explore more sensing modes (such as humidity, friction, etc.) and optimize the algorithm to adapt to more complex environments and application scenarios.

A multi-modal approach to continuous material identification through tactile sensing

Human-Touch-Inspired Material Recognition for Robotic Tactile Sensing

Design of a Biomimetic Tactile Sensor for Material Classification

Analyzing Material Recognition Performance of Thermal Tactile Sensing using a Large Materials Database and a Real Robot

Open‐Environment Tactile Sensing System: Towards Simple and Efficient Material Identification

Material Recognition Using Robotic Hand With Capacitive Tactile Sensor Array and Machine Learning

Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging

Surface material recognition through haptic exploration using an intelligent contact sensing finger

What Matters for Active Texture Recognition With Vision-Based Tactile Sensors

Artificial tactile perception smart finger for material identification based on triboelectric sensing

Bio-Inspired Magnetostrictive Tactile Sensor for Surface Material Recognition

Multimodal Material Classification Using Visual Attention

Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor

Magnetostrictive-based multimodal tactile sensors for object recognition

Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception

A flexible triboelectric tactile sensor for simultaneous material and texture recognition

A Thermosensitive Smart Robotic Self-Powered Sensor for Material Identification

Texture Recognition Based on Perception Data from a Bionic Tactile Sensor

Multifunctional Tactile Sensors for Object Recognition

Augmented Tactile Perception of Robotic Fingers Enabled by AI‐Enhanced Triboelectric Multimodal Sensors

Data-Driven Contact-Based Thermosensation for Enhanced Tactile Recognition