Abstract:The prosperous trend of deploying deep neural network (DNN) models to diverse hardware platforms has boosted the development of deep learning (DL) compilers. DL compilers take the high-level DNN model specifications as input and generate optimized DNN executables for diverse hardware architectures like CPUs, GPUs, and various hardware accelerators. Compiling DNN models into high-efficiency executables is not easy: the compilation procedure often involves converting high-level model specifications into several different intermediate representations (IR), e.g., graph IR and operator IR, and performing rule-based or learning-based optimizations from both platform-independent and platform-dependent perspectives. Despite the prosperous adoption of DL compilers in real-world scenarios, principled and systematic understanding toward the correctness of DL compilers does not yet exist. To fill this critical gap, this paper introduces MT-DLComp, a metamorphic testing framework specifically designed for DL compilers to effectively uncover erroneous compilations. Our approach leverages deliberately-designed metamorphic relations (MRs) to launch semantics-preserving mutations toward DNN models to generate their variants. This way, DL compilers can be automatically examined for compilation correctness utilizing DNN models and their variants without requiring manual intervention. We also develop a set of practical techniques to realize an effective workflow and localize identified error-revealing inputs. Real-world DL compilers exhibit a high level of engineering quality. Nevertheless, we detected over 435 inputs that can result in erroneous compilations in four popular DL compilers, all of which are industry-strength products maintained by Amazon, Facebook, Microsoft, and Google. While the discovered error-triggering inputs do not cause the DL compilers to crash directly, they can lead to the generation of incorrect DNN executables. With substantial manual effort and help from the DL compiler developers, we uncovered four bugs in these DL compilers by debugging them using the error-triggering inputs. Our proposed testing frameworks and findings can be used to guide developers in their efforts to improve DL compilers.

REDLC: Learning-driven Reverse Engineering for Deep Learning Compilers

DeepTheft: Stealing DNN Model Architectures through Power Side Channel

Model-less Is the Best Model: Generating Pure Code Implementations to Replace On-Device DL Models

Revisiting Deep Learning for Variable Type Recovery

Metamorphic Testing of Deep Learning Compilers

IRaDT: LLVM IR as Target for Efficient Neural Decompilation

Enhancing Reverse Engineering: Investigating and Benchmarking Large Language Models for Vulnerability Analysis in Decompiled Binaries

The Deep Learning Compiler: A Comprehensive Survey

Precise Extraction of Deep Learning Models via Side-Channel Attacks on Edge/Endpoint Devices

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing

New Security Challenges on Machine Learning Inference Engine: Chip Cloning and Model Reverse Engineering

Mind Your Heart: Stealthy Backdoor Attack on Dynamic Deep Neural Network in Edge Computing

Neural reverse engineering of stripped binaries using augmented control flow graphs

LSP Framework: A Compensatory Model for Defeating Trigger Reverse Engineering via Label Smoothing Poisoning

How to Steer Your Adversary: Targeted and Efficient Model Stealing Defenses with Gradient Redirection

RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking

Red Teaming Language Model Detectors with Language Models

DLP: towards active defense against backdoor attacks with decoupled learning process

Data Stealing Attacks against Large Language Models via Backdooring

Model-Reuse Attacks on Deep Learning Systems

DeCE: Deceptive Cross-Entropy Loss Designed for Defending Backdoor Attacks