Convolutional Neural Networks and Raman Spectroscopy for Semi-Supervised Dataset Construction and Transfer Learning Applications in Real-Time Quantitative Detection

Yuan Liu,Xiaotian Zhou,Xiaopeng Zhang,Jian Wang,Zhenguo Wen
DOI: https://doi.org/10.1145/3665689.3665706
2024-01-01
Abstract:Online Raman spectroscopy technology uses frequency shift of scattered light to obtain real-time characteristic peak information of the test substance, combined with machine learning regression models for real-time concentration prediction. However, in the microbial cultivation process, due to the complex and variable composition of the cultivation system substances, this poses a great challenge to the performance of the model. In addition, model training requires a large amount of data, and the process of collecting a large number of samples for offline detection is very difficult. In this study, a convolutional neural network was introduced to replace the partial least squares model in chemometrics to improve the accuracy of model prediction, and a non-offline sampling labeling technique was proposed to obtain a semi-supervised training set with sufficient data. For multiple detection substances, each substance's feature peak position was optimized using a genetic algorithm, and transfer learning was implemented through fine-tuning model structure and hyperparameters to retrain difficult-to-sample substance correlation models. Introduction of transfer learning also adapts to errors in the production scaling-up process without the need for completely retraining the model. With its unique convolutional kernel structure, the convolutional neural network replaces the tedious sliding window filtering and calibration processes in spectral pre-processing, simplifying the spectral processing workflow. After multiple batches of data cross-validation training, the model exhibits excellent robustness and accuracy, and is suitable for predicting substances in microbial cultivation process.
What problem does this paper attempt to address?