Abstract:For decades, researchers have developed task-specific models to address scientific challenges across diverse disciplines. Recently, large language models (LLMs) have shown enormous capabilities in handling general tasks; however, these models encounter difficulties in addressing real-world scientific problems, particularly in domains involving large-scale numerical data analysis, such as experimental high energy physics. This limitation is primarily due to BPE tokenization's inefficacy with numerical data. In this paper, we propose a task-agnostic architecture, BBT-Neutron, which employs a binary tokenization method to facilitate pretraining on a mixture of textual and large-scale numerical experimental data. The project code is available at <a class="link-external link-https" href="https://github.com/supersymmetry-technologies/bbt-neutron" rel="external noopener nofollow">this https URL</a>. We demonstrate the application of BBT-Neutron to Jet Origin Identification (JoI), a critical categorization challenge in high-energy physics that distinguishes jets originating from various quarks or gluons. Our results indicate that BBT-Neutron achieves comparable performance to state-of-the-art task-specific JoI models. Furthermore, we examine the scaling behavior of BBT-Neutron's performance with increasing data volume, suggesting the potential for BBT-Neutron to serve as a foundational model for particle physics data analysis, with possible extensions to a broad spectrum of scientific computing applications for Big Science experiments, industrial manufacturing and spacial computing.

A data-driven and model-agnostic approach to solving combinatorial assignment problems in searches for new physics

Data-driven and model-agnostic approach to solving combinatorial assignment problems in searches for new physics

Scaling Particle Collision Data Analysis

GAN-AE : An anomaly detection algorithm for New Physics search in LHC data

PASCL: Supervised Contrastive Learning with Perturbative Augmentation for Particle Decay Reconstruction

Zero-bias new particle searches using autoencoders in UPCs and diffractive events

Improving smuon searches with Neural Networks

Learning New Physics from Data -- a Symmetrized Approach

Reconstructing axion-like particles from beam dumps with simulation-based inference

Searching for Exotic Particles in High-Energy Physics with Deep Learning

Machine Learning for New Physics Searches

Applying Machine Learning Techniques To Intermediate-Length Cascade Decays

Data-Directed Search for New Physics based on Symmetries of the SM

Anomaly Detection in Presence of Irrelevant Features

The interplay of machine learning-based resonant anomaly detection methods

Model-independent analysis of new physics effects in decay

Modeling of Energy Distributions in Pseudo-Rest Frame Analyses of Two-Body Decays with Missing Particles

RESuM: Rare Event Surrogate Model for Physics Detector Design

Advancing Physics Data Analysis through Machine Learning and Physics-Informed Neural Networks

A novel quantum machine learning classifier to search for new physics