Spectrum-BERT: Pre-training of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors

Yansong Wang,Yundong Sun,Yansheng Fu,Dongjie Zhu,Zhaoshuo Tian

DOI: https://doi.org/10.1109/tim.2024.3374300

IF: 5.6

2024-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Counterfeit Chinese liquor incidents in China have significantly disrupted market order and jeopardized the health of consumers. Currently, deep learning-based spectral detection techniques are extensively leveraged in non-invasive food inspection. Excessive reliance on labels severely limits its application in real scenarios. To make better use of limited samples, we are the first to use the “unsupervised pre-training & supervised fine-tuning” paradigm in combining the Transformer architecture for feature extraction and classification of the Chinese liquor spectrum, and propose Spectrum-BERT, which represents Bidirectional Encoder Representations from Transformers for Spectrum. Specifically, we creatively propose spectral curve partitioning and 1-D convolutional layer mapping to maintain the model’s sensitivity to characteristic peak locations and local information of spectral curves. Moreover, the paradigm of “unsupervised pre-training & supervised fine-tuning” addresses the limitation of label deficiency, thus improving the model’s applicability. Finally, we have conducted extensive experiments on the real liquor spectral dataset. Comparative experiments demonstrate that Spectrum-BERT outperforms all baselines on all metrics using only 70% supervised signal. The limit experimental results show that Spectrum-BERT can still maintain its lead using only the 10% supervised signal. Thanks to the more efficient model architecture, Spectrum-BERT’s model parameters and FLOPs are only 1/568 and 1/322 of those of baselines.

engineering, electrical & electronic,instruments & instrumentation