Abstract:Accurate classification of celestial objects is essential for advancing our understanding of the universe. MargNet is a recently developed deep learning-based classifier applied to the Sloan Digital Sky Survey (SDSS) Data Release 16 (DR16) dataset to segregate stars, quasars, and compact galaxies using photometric data. MargNet utilizes a stacked architecture, combining a Convolutional Neural Network (CNN) for image modelling and an Artificial Neural Network (ANN) for modelling photometric parameters. Notably, MargNet focuses exclusively on compact galaxies and outperforms other methods in classifying compact galaxies from stars and quasars, even at fainter magnitudes. In this study, we propose enhancing MargNet's performance by incorporating attention mechanisms and Vision Transformer (ViT)-based models for processing image data. The attention mechanism allows the model to focus on relevant features and capture intricate patterns within images, effectively distinguishing between different classes of celestial objects. Additionally, we leverage ViTs, a transformer-based deep learning architecture renowned for exceptional performance in image classification tasks. We enhance the model's understanding of complex astronomical images by utilizing ViT's ability to capture global dependencies and contextual information. Our approach uses a curated dataset comprising 240,000 compact and 150,000 faint objects. The models learn classification directly from the data, minimizing human intervention. Furthermore, we explore ViT as a hybrid architecture that uses photometric features and images together as input to predict astronomical objects. Our results demonstrate that the proposed attention mechanism augmented CNN in MargNet marginally outperforms the traditional MargNet and the proposed ViT-based MargNet models. Additionally, the ViT-based hybrid model emerges as the most lightweight and easy-to-train model with classification accuracy similar to that of the best-performing attention-enhanced MargNet. This advancement in deep learning will contribute to greater success in identifying objects in upcoming surveys like the Vera C. Rubin Large Synoptic Survey Telescope.

Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances

Paying Attention to Astronomical Transients: Introducing the Time-series Transformer for Photometric Classification

ATAT: Astronomical Transformer for time series And Tabular data

Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices

Uncertainty Quantification for Transformer Models for Dark-Pattern Detection

Application of Convolutional Neural Networks to time domain astrophysics. 2D image analysis of OGLE light curves

Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning

Detection, instance segmentation, and classification for astronomical surveys with deep learning ( deepdisc ): detectron2 implementation and demonstration with Hyper Suprime-Cam data

Enhanced astronomical source classification with integration of attention mechanisms and vision transformers

Periodic Variable Star Classification with Deep Learning: Handling Data Imbalance in an Ensemble Augmentation Way

Real-time Detection of Anomalies in Multivariate Time Series of Astronomical Data

Calibrating Transformers via Sparse Gaussian Processes

Astronomical image time series classification using CONVolutional attENTION (ConvEntion)

Identifying Light-curve Signals with a Deep-learning-based Object Detection Algorithm. II. A General Light-curve Classification Framework

Rough Transformers for Continuous and Efficient Time-Series Modelling

Transformation Based Deep Anomaly Detection in Astronomical Images

Domain Adaptation via Minimax Entropy for Real/Bogus Classification of Astronomical Alerts

Explainable classification of astronomical uncertain time series

Automated classification of remote sensing satellite images using deep learning based vision transformer