Abstract:s Accurately identifying grain types in thin sections of sandy sediments or sandstones is crucial for understanding their provenance, depositional environments, and potential as natural resources. Although traditional computer vision methods and machine learning algorithms have been used for automatic grain identification, recent advancements in deep learning techniques have opened up new possibilities for achieving more reliable results with less manual labor. In this study, we present Trans-SedNet, a state-of-the-art dual-modal Vision-Transformer (ViT) model that uses both cross- (XPL) and plane-polarized light (PPL) images to achieve semantic segmentation of thin-section images. Our model classifies a total of ten grain types, including subtypes of quartz, feldspar, and lithic fragments, to emulate the manual identification process in sedimentary petrology. To optimize performance, we use SegFormer as the model backbone and add window- and mix-attention to the encoder to identify local information in the images and to best use XPL and PPL images. We also use a combination of focal and dice loss and a smoothing procedure to address imbalances and reduce over-segmentation. Our comparative analysis of several deep convolution neural networks and ViT models, including FCN, U-Net, DeepLabV3Plus, SegNeXT, and CMX, shows that Trans-SedNet outperforms the other models with a significant increase in evaluation metrics of mIoU and mPA. We also conduct an experiment to test the models' ability to handle dual-modal information, which reveals that the dual-modal models, including Trans-SedNet, achieve better results than single-modal models with the extra input of PPL images. Our study demonstrates the potential of ViT models in semantic segmentation of thin-section images and highlights the importance of dual-modal models for handling complex input in various geoscience disciplines. By improving data quality and quantity, our model has the potential to enhance the efficiency and reliability of grain identification in sedimentary petrology and relevant subjects.

Spectraformer: deep learning model for grain spectral qualitative analysis based on transformer structure

Powdery Food Identification Using NIR Spectroscopy and Extensible Deep Learning Model

DeepSpectra: an End-to-end Deep Learning Approach for Quantitative Spectral Analysis

A Portable NIR-system for Mixture Powdery Food Analysis Using Deep Learning

SpectraTr: A novel deep learning model for qualitative analysis of drug spectroscopy based on transformer structure

Application of Segmented Transformer Feature Extraction in Near Infrared Spectral Data Classification

The spectral resolution of unknown mixture based on THz spectroscopy with self-modeling technique

End‐to‐end analysis modeling of vibrational spectroscopy based on deep learning approach

Integrating portable NIR spectrometry with deep learning for accurate Estimation of crude protein in corn feed

DsTer: A dense spectral transformer for remote sensing spectral super-resolution

Development of Wheat Component Detector Based on Near Infrared Spectrum

Sediment grain segmentation in thin-section images using dual-modal Vision Transformer

Deep learning-based Raman spectroscopy qualitative analysis algorithm: A convolutional neural network and transformer approach

Wheat Kernel Variety Identification Based on a Large Near-Infrared Spectral Dataset and a Novel Deep Learning-Based Feature Selection Method

Fusion of convolutional neural network with XGBoost feature extraction for predicting multi-constituents in corn using near infrared spectroscopy

An interpretable deep learning approach for calibration transfer among multiple near-infrared instruments

Application of Near-Infrared Hyperspectral Imaging for Variety Identification of Coated Maize Kernels with Deep Learning

Markov Transition Field Combined with Convolutional Neural Network Improved the Predictive Performance of Near-Infrared Spectroscopy Models for Determination of Aflatoxin B1 in Maize

SpectraFM: Tuning into Stellar Foundation Models

1D-Inception-Resnet for NIR quantitative analysis and its transferability between different spectrometers

Discrimination of New and Aged Seeds Based on On-Line Near-Infrared Spectroscopy Technology Combined with Machine Learning