Abstract:As the quantum computing community gravitates towards understanding the practical benefits of quantum computers, having a clear definition and evaluation scheme for assessing practical quantum advantage in the context of specific applications is paramount. Generative modeling, for example, is a widely accepted natural use case for quantum computers, and yet has lacked a concrete approach for quantifying success of quantum models over classical ones. In this work, we construct a simple and unambiguous approach to probe practical quantum advantage for generative modeling by measuring the algorithm's generalization performance. Using the sample-based approach proposed here, any generative model, from state-of-the-art classical generative models such as generative adversarial networks (GANs) to quantum models such as quantum circuit born machines, can be evaluated under the same conditions on a concrete well-defined framework. In contrast to other sample-based metrics for probing practical generalization, we leverage constrained optimization problems (e.g., cardinality-constrained problems) and use these discrete datasets to define specific metrics capable of unambiguously measuring the quality of the samples and the model's generalization capabilities for generating data beyond the training set, but still within the valid solution space. Additionally, our metrics can diagnose trainability issues such as mode collapse and overfitting, as we illustrate when comparing GANs to quantum-inspired models built out of tensor networks. Our simulation results show that our quantum-inspired models have up to a 68 times enhancement in generating unseen unique and valid samples compared to the GANs explored here, and a ratio of 61:2 for generating samples with better quality than those observed in the training set. We foresee these metrics as valuable tools for rigorously defining practical quantum advantage in the domain of generative modeling, where finding a winning model remains an ongoing challenge as new architectures are developed. https://doi.org/10.1103/PhysRevApplied.21.044032 © 2024 American Physical Society

A Quantitative Measure of Theoretical Scientific Merit

A quantitative analysis of measures of quality in science

A measure for the impact of research

Evaluation Measures for Quantification: An Axiomatic Approach

A scientists' view of scientometrics: Not everything that counts can be counted

Theoretical research without projects

Quantifying Intuition: Bayesian Approach to Figures of Merit in EXAFS Analysis of Magic Size Clusters

A theory and methodology to quantify knowledge

A framework for the measurement and prediction of an individual scientist's performance

The Assessment of Science: The Relative Merits of Post-Publication Review, the Impact Factor, and the Number of Citations

A figure of merit that includes 5 distinct performance indicators to improve research evaluation of academic scholars'

Distance-based resource quantification for sets of quantum measurements

Measuring Scientific Broadness

A Model of Classical and Quantum Measurement

Extending the Q-score to an Application-level Quantum Metric Framework

Quantifying Success in Science: An Overview

Generalization metrics for practical quantum advantage in generative models

Scientific and Financial Performance Measure : A Simultaneous Model to Evaluate Scientific Activities

Evaluating Three Levels of Quantum Metrics on Quantum-Inspire Hardware

Measurement Theory in the Philosophy of Science

QuAS: Quantum Application Score for benchmarking the utility of quantum computers