Abstract:The QED-C suite of Application-Oriented Benchmarks provides the ability to gauge performance characteristics of quantum computers as applied to real-world applications. Its benchmark programs sweep over a range of problem sizes and inputs, capturing key performance metrics related to the quality of results, total time of execution, and quantum gate resources consumed. In this manuscript, we investigate challenges in broadening the relevance of this benchmarking methodology to applications of greater complexity. First, we introduce a method for improving landscape coverage by varying algorithm parameters systematically, exemplifying this functionality in a new scalable HHL linear equation solver benchmark. Second, we add a VQE implementation of a Hydrogen Lattice simulation to the QED-C suite, and introduce a methodology for analyzing the result quality and run-time cost trade-off. We observe a decrease in accuracy with increased number of qubits, but only a mild increase in the execution time. Third, unique characteristics of a supervised machine-learning classification application are explored as a benchmark to gauge the extensibility of the framework to new classes of application. Applying this to a binary classification problem revealed the increase in training time required for larger anzatz circuits, and the significant classical overhead. Fourth, we add methods to include optimization and error mitigation in the benchmarking workflow which allows us to: identify a favourable trade off between approximate gate synthesis and gate noise; observe the benefits of measurement error mitigation and a form of deterministic error mitigation algorithm; and to contrast the improvement with the resulting time overhead. Looking ahead, we discuss how the benchmark framework can be instrumental in facilitating the exploration of algorithmic options and their impact on performance.

Application-Aware Benchmarking of NISQ Hardware

Scalable Benchmarks for Gate-Based Quantum Computers

QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation

Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks

Clifford-based Circuit Cutting for Quantum Simulation

Benchmarking Quantum Circuit Transformation with QKNOB Circuits

Benchmarking simulated and physical quantum processing units using quantum and hybrid algorithms

Sparse Simulation of VQE Circuits for Quantum Chemistry

Benchmarking digital quantum simulations above hundreds of qubits using quantum critical dynamics

Establishing the Quantum Supremacy Frontier with a 281 Pflop/s Simulation

Quantum Volume in Practice: What Users Can Expect from NISQ Devices

Comparison of Superconducting NISQ Architectures

Scalable Full-Stack Benchmarks for Quantum Computers

Benchmarking the performance of quantum computing software

Enabling Multi-programming Mechanism for Quantum Computing in the NISQ Era

Benchmarking near-term quantum computers via random circuit sampling

Quantum Chemistry as a Benchmark for Near-Term Quantum Computers

Benchmarking bosonic and fermionic dynamics

Queen: A quick, scalable, and comprehensive quantum circuit simulation for supercomputing

Noise Aware Utility Optimization of NISQ Devices

Benchmarking an 11-qubit quantum computer