Abstract:Deep Learning (DL) libraries, such as PyTorch, are widely used for building and deploying DL models on various hardware platforms. Meanwhile, they are found to contain bugs that lead to incorrect calculation results and cause issues like non-convergence training and inaccurate prediction of DL models. Thus, many efforts have been made to test DL libraries and reveal bugs. However, existing DL library testing methods manifest limitations: model-level testing methods cause complexity in fault localization. Meanwhile, API-level testing methods often generate invalid inputs or primarily focus on extreme inputs that lead to crash failures; they also ignore testing realistic API interactions. These limitations may lead to missing detection of bugs, even in the frequently used APIs. To address these limitations, we propose SORT (Subgraph-Oriented Realistic Testing) to differential test DL libraries on different hardware platforms. SORT takes popular API interaction patterns, represented as frequent subgraphs of model computation graphs, as test subjects. In this way, it introduces realistic API interaction sequences while maintaining efficiency in locating faulty APIs for observed errors. Besides, SORT prepares test inputs by referring to extensive features of runtime inputs for each API in executing real-life benchmark data. The generated inputs are expected to better simulate such valid real inputs and reveal bugs more likely to happen in real-life usage. Evaluation on 728 frequent subgraphs of 49 popular PyTorch models demonstrates that SORT achieves a 100\% valid input generation rate, detects more precision bugs than existing methods, and reveals interaction-related bugs missed by single-API testing. 18 precision bugs in PyTorch are identified.

Testing Graph Database Systems with Graph-State Persistence Oracle

Effective Bug Detection in Graph Database Engines: An LLM-based Approach

Dinkel: Testing Graph Database Engines via State-Aware Query Generation

GDsmith: Detecting Bugs in Graph Database Engines

Detecting DBMS Bugs with Context-Sensitive Instantiation and Multi-Plan Execution

Detecting Transactional Bugs in Database Engines via Graph-Based Oracle Construction

Testing Database Engines via Query Plan Guidance

A Demonstration of DLBD: Database Logic Bug Detection System.

DynSQL: Stateful Fuzzing for Database Management Systems with Complex and Valid SQL Query Generation

Detecting Logic Bugs of Join Optimizations in DBMS.

G-Tran: A High Performance Distributed Graph Database with a Decentralized Architecture

Subgraph-Oriented Testing for Deep Learning Libraries

GLIB: Towards Automated Test Oracle for Graphically-Rich Applications

Uncovering Bugs In Distributed Storage Systems During Testing (Not In Production!)

Testing Constraint Checking Implementations Via Principled Metamorphic Transformations

Mozi: Discovering DBMS Bugs Via Configuration-Based Equivalent Transformation

Building a High-Performance Graph Storage on Top of Tree-Structured Key-Value Stores

Conformance Testing of Relational DBMS Against SQL Specifications

Griffin : Grammar-Free DBMS Fuzzing.

G-SQL: Fast Query Processing via Graph Exploration.

Finding Logic Bugs in Spatial Database Engines via Affine Equivalent Inputs