Abstract:Existing tools for the automated detection of memory corruption bugs are not very effective in practice. They typically recognize only standard memory management (MM) APIs (e.g., malloc and free) and assume a naive paired-use model—an allocator is followed by a specific deallocator. However, we observe that programmers very often design their own MM functions and that these functions often manifest two major characteristics: (1) Custom allocator functions perform multi-object or nested allocation which then requires structure-aware deallocation functions. (2) Custom allocators and deallocators follow an unpaired-use model. A more effective detection thus needs to adapt those characteristics and capture memory bugs related to non-standard MM behaviors. In this paper, we present a MM function aware memory bug detection technique by introducing the concept of structure-aware and object-centric Memory Operation Synopsis (MOS). A MOS abstractly describes the memory objects of a given MM function, how they are managed by the function, and their structural relations. By utilizing MOS, a bug detection could explore much less code but is still capable of handling multi-object or nested allocations and does not rely on the paired-use model. In addition, to extensively find MM functions and automatically generate MOS for them, we propose a new identification approach that combines natural language processing (NLP) and data flow analysis, which enables the efficient and comprehensive identification of MM functions, even in very large code bases. We implement a MOS-enhanced memory bug detection system, Goshawk, to discover memory bugs caused by complex and custom MM behaviors. We applied Goshawk to well-tested and widely-used open source projects including OS kernels, server applications, and IoT SDKs. Goshawk outperforms the state-of-the-art data flow analysis driven bug detection tools by an order of magnitude in analysis speed and the number of accurately identified MM functions, reports the discovered bugs with a developer-friendly, MOS based description, and successfully detects 92 new double-free and use-after-free bugs.

Mtool: an integrated system for performance debugging shared memory multiprocessor applications

Log-based Architectures for General-Purpose Monitoring of Deployed Code

A tool for top-down performance analysis of GPU-accelerated applications

Jdebug: A Fast, Non-intrusive and Scalable Fault Locating Tool for Ten-Million-Scale Parallel Applications

A Lightweight Hybrid Hardware/software Approach for Object-Relative Memory Profiling

Design and Implementation of Parallel Debugger and Profiler for MPJ Express

Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems

Parallel Computing of Shared Memory Multiprocessors Based on JOMP

Automatic parallelization of fine-grained metafunctions on a chip multiprocessor

Understanding object-level memory access patterns across the spectrum

A Source-Level Instrumentation Framework for the Dynamic Analysis of Memory Safety

Evaluation and Design Trade-Offs Between Symmetric Shared-Memory and Distributed Shared-Memory

Performance Analysis and Optimization of a Hybrid Distributed Reverse Time Migration Application

HMTT: A Hybrid Hardware/Software Tracing System for Bridging Memory Trace's Semantic Gap

Spindle: Informed Memory Access Monitoring

A Learning Support Method for Multi-threaded Programs Using Trace Tables

Moped: Orchestrating Interprocess Message Data On Cmps

MOOSE Stochastic Tools: A module for performing parallel, memory-efficient in situ stochastic simulations

Goshawk: Hunting Memory Corruptions Via Structure-Aware and Object-Centric Memory Operation Synopsis

Lossless instruction-to-object memory tracing in the Linux kernel

Rule (4) and Continuous Observation