Abstract:An appropriate pretreatment method is crucial to establish a reliable model for near-infrared spectroscopy. It is usually selected by comparing model performance with different pretreatment methods, such as the root mean square error of internal cross-validation or the root mean square error of prediction. When the statistical indexes of the models are similar and indistinguishable, or the results of model calibration and cross-validation are too different, indicating an overfitting situation, it may be due to the selection of an inappropriate pretreatment. In this paper, an approach is proposed to select appropriate pretreatment methods by chemical components valid and invalid absorption wavebands. Due to there being no chemical groups absorption in wavebands of 1250–1350 nm and 1780–1850 nm, they are considered chemical invalid absorption bands. The root mean square distance in the invalid wavebands characterizes the difference in samples' physical state, such as particle size and instrument error. Except for the invalid absorption band, other wavebands in the near-infrared region are considered chemical valid absorption wavebands. The root mean square distance in the valid absorption wavebands mainly characterize the chemical composition differences between different samples. So, we define effective information rate of spectra. The larger effective information rate of spectra, the more favorable it is to establish a quantitative model. In this study, spectra of 60 wheat powder samples were used to compare seven pretreatment methods. Moreover, the average effective information rates of spectra were calculated for the seven pretreatment methods respectively. The results show that the combination of standard normal variate transformation and first derivative with Savitzky–Golay smoothing is the best. Finally, a comparison of model prediction with different pretreatment methods was made. The results are the same as the former. Therefore, using the effective information rate of spectra for selecting an appropriate near-infrared spectroscopy pretreatment method is practical and can be used not only for near-infrared modeling but also for designing near-infrared instruments as an index for optimization.

A local pre-processing method for near-infrared spectra, combined with spectral segmentation and standard normal variate transformation

Spectral-information-divergence based local pls modeling algorithm in near infrared spectroscopy

Local Regression Algorithm Based on Net Analyte Signal and Its Application in Near Infrared Spectral Analysis

[A Simple and Convenient Standardization Algorithm of NIR Spectra].

Preprocessing methods for near‐infrared spectrum calibration

Robust Local Pcr and Its Application in Nir Spectral Quantitative Analysis

[Near-Infrared Spectral Quantitative Analysis by Combining Classification with Local PLS].

Self-expansion full information optimization strategy: Convenient and efficient method for near infrared spectrum auto-analysis

Selecting near-infrared reflection spectroscopy pretreatment methods by chemical components valid and invalid absorption wavebands

[Research Progress on Method Validation of Near Infrared Spectroscopy in Quantitative Analysis].

Application of Spectrum Preprocessing Algorithms in Optimizing NIR Models of Agricultural Products

On-line Monitoring of Traditional Chinese Medicinal Powder Blending Process by Using Near-Infrared Spectroscopy

[Research Progress on Standardization Study of NIR Spectroscopy Based Method for Quality Control of Traditional Chinese Medicine].

New approach for near-infrared wavelength selection using a combination of MIC and firefly evolution

A three-stage wavelength selection algorithm for near-infrared spectroscopy calibration

NIR Spectroscopy As a Process Analytical Technology (PAT) Tool for On-Line and Real-Time Monitoring of an Extraction Process

Algorithm of Stability-Analysis-Based Feature Selection for NIR Calibration Transfer

The use of computer modeling techniques in near infrared spectroscopy

Aggregated functional data model for Near-Infrared Spectroscopy calibration and prediction

A Review of Machine Learning for Near-Infrared Spectroscopy

Optical Coefficient‐based Multivariate Calibration on Near‐infrared Spectroscopy