Abstract:There has been a recent explosion in research into machine-learning-based generative modeling to tackle computational challenges for simulations in high energy physics (HEP). In order to use such alternative simulators in practice, we need well-defined metrics to compare different generative models and evaluate their discrepancy from the true distributions. We present the first systematic review and investigation into evaluation metrics and their sensitivity to failure modes of generative models, using the framework of two-sample goodness-of-fit testing, and their relevance and viability for HEP. Inspired by previous work in both physics and computer vision, we propose two new metrics, the Fréchet and kernel physics distances (FPD and KPD, respectively) and perform a variety of experiments measuring their performance on simple Gaussian-distributed and simulated high energy jet datasets. We find FPD, in particular, to be the most sensitive metric to all alternative jet distributions tested and recommend its adoption, along with the KPD and Wasserstein distances between individual feature distributions, for evaluating generative models in HEP. We finally demonstrate the efficacy of these proposed metrics in evaluating and comparing a novel attention-based generative adversarial particle transformer to the state-of-the-art message-passing generative adversarial network jet simulation model. The code for our proposed metrics is provided in the open source j et n et python library. https://doi.org/10.1103/PhysRevD.107.076017 Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI. Funded by SCOAP 3 . Published by the American Physical Society

Differentiable vertex fitting for jet flavor tagging

Differentiable Vertex Fitting for Jet Flavour Tagging

Jet Flavor Classification in High-Energy Physics with Deep Neural Networks

Jet Flavour Tagging at FCC-ee with a Transformer-based Neural Network: DeepJetTransformer

From strange-quark tagging to fragmentation tagging with machine learning

Flavoured jets with exact anti-$k_t$ kinematics and tests of infrared and collinear safety

Practical Jet Flavour Through NNLO

Vertex Reconstruction with MaskFormers

Inferring flavor mixtures in multijet events

Secondary vertex reconstruction with MaskFormers

A dress of flavour to suit any jet

The Fundamental Limit of Jet Tagging

PCN: A Deep Learning Approach to Jet Tagging Utilizing Novel Graph Construction Methods and Chebyshev Graph Convolutions

Efficient and Robust Jet Tagging at the LHC with Knowledge Distillation

Pipeline for performance evaluation of flavour tagging dedicated Graph Neural Network algorithms

Variational Pseudo Marginal Methods for Jet Reconstruction in Particle Physics

A modern framework for jet tagger development

Exploring jets: substructure and flavour tagging in CMS and ATLAS

Quark Jet Versus Gluon Jet: Fully-Connected Neural Networks with High-Level Features

Evaluating generative models in high energy physics