A Tool for Estimating Memory Usage

Shengyi Wang,Zongyan Qiu

DOI: https://doi.org/10.1109/TASE.2009.39

2009-01-01

Abstract:We introduce a tool under development for assembly-level programs, which captures memory requirement of each method in terms of symbolic values of its parameters. It exploits fix-point analysis for loops and recursions. Up now the tool can handle most common structures in programs except some rare situations.

What problem does this paper attempt to address?

Stack Bound Inference for Abstract Java Bytecode.

Shengyi Wang,Zongyan Qiu,Shengchao Qin,Wei-Ngan Chin

DOI: https://doi.org/10.1109/tase.2010.24

2010-01-01

Abstract:Ubiquitous embedded systems are often resource-constrained. Developing software for these systems should take into account resources such as memory space. In this paper, we develop and implement an analysis framework to infer statically stack usage bounds for assembly-level programs in abstract Java Byte code. Our stack bound inference process, extended from a theoretical framework proposed earlier by some of the authors, is composed of deductive inference rules in multiple passes. Based on these rules, a usable tool has been developed for processing programs to capture the stack memory needs of each procedure in terms of the symbolic values of its parameters. The final result contains path-sensitive information to achieve better precision. The tool invokes a Presburger solver to perform fixed point analysis for loops and recursive procedures. Our initial experiments have confirmed the viability and power of the approach.
A Dynamic Approach to Detecting, Eliminating and Fixing Memory Leaks

Bin Yu,Cong Tian,Nan Zhang,Zhenhua Duan,Hongwei Du

DOI: https://doi.org/10.1007/s10878-019-00398-x

2019-01-01

Journal of Combinatorial Optimization

Abstract:This paper presents a dynamic approach to detecting, eliminating and fixing memory leaks. With our approach, a program to be analyzed is instrumented before its execution. Dynamic symbolic execution is employed so as to expose memory leaks occurring in all execution paths. During the program execution, information about each allocated memory area is updated when corresponding statements are executed. Based on this information, a back-end leak checker records the changes of variables pointing to each memory area, detects memory leaks and deallocates leaked memory areas. After the program execution, a fixed program containing deallocation statements is generated. We have implemented our approach in a tool called DEF_LEAK based on Low Level Virtual Machine compiler infrastructure. Experiments show that DEF_LEAK can detect more leaks than the existing dynamic detection tools for programs with user inputs. Moreover, DEF_LEAK is more effective for helping programmers understand and fix memory leaks than other detection tools.
Benchmark of Tools for Memory Leak

李倩,潘敏学,李宣东

DOI: https://doi.org/10.3778/j.issn.1673-9418.2010.01.003

2010-01-01

Abstract:Memory leak,which is a substantial source of errors in software systems,consumes system memory,degrading performance and eventually resulting in program crashes.There are two kinds of tools for detecting memory leaks,which are static tools based on program static analysis technique,and dynamic tools recording allocation of heaps during program running.So far,benchmarks for evaluating their ability are still nonexistent.An approach to build this benchmark is proposed,starting from comprehension and recognition of memory leak and ability of those detecting tools.
Spindle: Informed Memory Access Monitoring

Haojie Wang,Jidong Zhai,Xiongchao Tang,Bowen Yu,Xiaosong Ma,Wenguang Chen

2018-01-01

Abstract:Memory monitoring is of critical use in understanding applications and evaluating systems. Due to the dynamic nature in programs' memory accesses, common practice today leaves large amounts of address examination and data recording at runtime, at the cost of substantial performance overhead (and large storage time/space consumption if memory traces are collected). Recognizing the memory access patterns available at compile time and redundancy in runtime checks, we propose a novel memory access monitoring and analysis framework, Spindle. Unlike methods delaying all checks to runtime or performing task-specific optimization at compile time, Spindle performs common static analysis to identify predictable memory access patterns into a compact program structure summary. Custom memory monitoring tools can then be developed on top of Spindle, leveraging the structural information extracted to dramatically reduce the amount of instrumentation that incurs heavy runtime memory address examination or recording. We implement Spindle in the popular LLVM compiler, supporting both single-thread and multi-threaded programs. Our evaluation demonstrated the effectiveness of two Spindle-based tools, performing memory bug detection and trace collection respectively, with a variety of programs. Results show that these tools are able to aggressively prune online memory monitoring processing, fulfilling desired tasks with performance overhead significantly reduced (2:54× on average for memory bug detection and over 200× on average for access tracing, over state-of-the-art solutions).
TsmartGP: a tool for finding memory defects with pointer analysis

Yuexing Wang,Guang Chen,Min Zhou,Ming Gu,Jiaguang Sun

DOI: https://doi.org/10.1109/ASE.2019.00129

2019-01-01

Abstract:ABSTRACTPrecise pointer analysis is desired since it is a core technique to find memory defects. There are several dimensions of pointer analysis precision, flow sensitivity, context sensitivity, field sensitivity and path sensitivity. For static analysis tools utilizing pointer analysis, considering all dimensions is difficult because the trade-off between precision and efficiency should be balanced. This paper presents TsmartGP, a static analysis tool for finding memory defects in C programs with a precise and efficient pointer analysis. The pointer analysis algorithm is flow, context, field, and quasi path sensitive. Control flow automatons are the key structures for our analysis to be flow sensitive. Function summaries are applied to get context information and elements of aggregate structures are handled to improve precision. Path conditions are used to filter unreachable paths. For efficiency, a multi-entry mechanism is proposed. Utilizing the pointer analysis algorithm, we implement a checker in TsmartGP to find uninitialized pointer errors in 13 real-world applications. Cppcheck and Clang Static Analyzer are chosen for comparison. The experimental results show that TsmartGP can find more errors while its accuracy is also higher than Cppcheck and Clang Static Analyzer. The demo video is available at https://youtu.be/IQlshemk6OA.
Who Ate My Memory? Towards Attribution in Memory Management

Gunnar Kudrjavets,Ayushi Rastogi,Jeff Thomas,Nachiappan Nagappan

DOI: https://doi.org/10.1109/ICSE-SEIP58684.2023.00040

2022-12-23

Abstract:To understand applications' memory usage details, engineers use instrumented builds and profiling tools. Both approaches are impractical for use in production environments or deployed mobile applications. As a result, developers can gather only high-level memory-related statistics for deployed software. In our experience, the lack of granular field data makes fixing performance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutions to collect detailed data about applications' memory usage to increase developer productivity. Current research into memory attribution-related data structures, techniques, and tools is in the early stages and enables several new research avenues.

Software Engineering
SCAD: Controlled Memory Allocation Analysis and Detection

Xiao Qixue,Chen Yu,Xu Yongjian,Mao Junjie,Guo Shize,Wang Zhenjiang,Shi Yuanchun

DOI: https://doi.org/10.1109/compsac.2015.20

2015-01-01

Abstract:Memory errors have been one of the main causes for software vulnerability. This paper discusses an issue called controlled memory allocation (CMA) which occurs when key elements of memory allocation are affected by elaborately designed input data. This paper proposes a renovated approach of CMA detection, utilizing static analysis, and optimizing symbolic execution system with path-guided technologies. Combining these technologies with the state-of-the-art symbolic execution engine, KLEE, we present a prototype CMA detection tool, SCAD. SCAD was tested on commonly used applications like Coreutils and Texinfo, where it found 14 CMA related bugs including 5 unknown previously ones. SCAD's path guided searcher could reach an assigned target faster and with more paths than other path searchers which are provided by KLEE. Two memory allocation sites in Coreutils could not be reached by 8 path searchers provided by KLEE in five minutes, but SCAD's path guided searcher could reach them in 24 seconds and 17 seconds respectively. For memory allocation related code, SCAD executes faster with higher coverage than conventional symbolic execution engines.
Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems

Ashwin Poduval,Hayden Coffey,Michael Swift

2024-11-17

Abstract:Memory performance is often the main bottleneck in modern computing systems. In recent years, researchers have attempted to scale the memory wall by leveraging new technology such as CXL, HBM, and in- and near-memory processing. Developers optimizing for such hardware need to understand how target applications perform to fully take advantage of these systems. Existing software and hardware performance introspection techniques are ill-suited for this purpose due to one or more of the following factors: coarse-grained measurement, inability to offer data needed to debug key issues, high runtime overhead, and hardware dependence. The heightened integration between compute and memory in many proposed systems offers an opportunity to extend compiler support for this purpose. We have developed Examem, a memory performance introspection framework based on the LLVM compiler infrastructure. Examem supports developer annotated regions in code, allowing for targeted instrumentation of kernels. Examem supports hardware performance counters when available, in addition to software instrumentation. It statically records information about the instruction mix of the code and adds dynamic instrumentation to produce estimated memory bandwidth for an instrumented region at runtime. This combined approach keeps runtime overhead low while remaining accurate, with a geomean overhead under 10% and a geomean byte accuracy of 93%. Finally, our instrumentation is performed using an LLVM IR pass, which is target agnostic, and we have applied it to four ISAs.

Performance,Hardware Architecture
A Source-Level Instrumentation Framework for the Dynamic Analysis of Memory Safety

Zhe Chen,Qi Zhang,Jun Wu,Junqi Yan,Jingling Xue

DOI: https://doi.org/10.1109/tse.2022.3210580

IF: 7.4

2023-04-21

IEEE Transactions on Software Engineering

Abstract:Low-level control makes C unsafe, resulting in memory errors that can lead to data corruption, security vulnerabilities or program crashes. Dynamic analysis tools, which have been widely used for detecting memory errors at runtime, usually perform instrumentation at the IR or binary level. However, these non-source-level instrumentation frameworks and tools suffer from two inherent drawbacks: optimization sensitivity and platform dependence. Due to optimization sensitivity, the user of these tools must trade either performance for effectiveness by compiling the program at -O0 or effectiveness for performance by compiling the program at a higher optimization level, say, -O3. In this paper, we propose a new source-level instrumentation framework to overcome these two drawbacks, and implement it in a new dynamic analysis tool, called Movec, that adopts a pointer-based monitoring algorithm. We have evaluated Movec comprehensively by using the NIST's SARD benchmark suite (1152 programs), a set of 126 microbenchmarks (with ground truth), a set of 20 MiBench benchmarks and 5 pure-C SPEC CPU 2017 benchmarks. In terms of effectiveness, Movec outperforms three state-of-the-art dynamic analysis tools, AddressSanitizer, SoftBoundCETS and Valgrind, for all the standard optimization levels (from -O0 to -O3). In terms of performance, Movec outperforms SoftBoundCETS and Valgrind, and is slower than AddressSanitizer but consumes less memory.

engineering, electrical & electronic,computer science, software engineering
A Lightweight Hybrid Hardware/software Approach for Object-Relative Memory Profiling

Licheng Chen,Zehan Cui,Yungang Bao,Mingyu Chen,Yongbing Huang,Guangming Tan

DOI: https://doi.org/10.1109/ispass.2012.6189205

2012-01-01

Abstract:Memory profiling is the process of collecting memory address traces during the execution of a program, then analyzing and characterizing the memory behavior of the program offline. With the trend that there will be more and more cores integrated in a processor chip, the “Memory Wall” problem will become more serious in the chip multiprocessor (CMP) system. Thus accurate and effective memory profiling is becoming one of the keys to identify the source of memory system bottlenecks. A large body of work has been contributed to memory profiling, however, most adopts instrumentation, simulator which suffers heavy overhead, or hardware performance counter which is lack of detail trace information. Furthermore, correlating the raw memory address traces with object-relative information allows us to separate regular pattern for certain object from the irregular mixed, thus helps the optimization. In this paper, we propose a lightweight hybrid hardware/software approach for object-relative memory profiling. We monitor physical memory addresses through hardware snooping with negligible overhead; meanwhile we dump Linux kernel page tables of processes, as well as object-relative memory allocation information. Our approach supports not only to collect applications' full memory traces with detail object relative information, but also to identify hardware-generated memory accesses such as page memory walks due to TLB miss at object level. The experimental results on real system show that our approach is highly accurate (the largest error is 2.04%) and low overhead (the average overhead is 1.60%). Furthermore, we profile two multi-thread applications in detail, and successfully identity hot TLB-miss objects. With object-targeted optimization, we can improve applications' performance by nearly 6.86%.
LLVM Static Analysis for Program Characterization and Memory Reuse Profile Estimation

Atanu Barai,Nandakishore Santhi,Abdur Razzak,Stephan Eidenbenz,Abdel-Hameed A. Badawy

2023-11-21

Abstract:Profiling various application characteristics, including the number of different arithmetic operations performed, memory footprint, etc., dynamically is time- and space-consuming. On the other hand, static analysis methods, although fast, can be less accurate. This paper presents an LLVM-based probabilistic static analysis method that accurately predicts different program characteristics and estimates the reuse distance profile of a program by analyzing the LLVM IR file in constant time, regardless of program input size. We generate the basic-block-level control flow graph of the target application kernel and determine basic-block execution counts by solving the linear balance equation involving the adjacent basic blocks' transition probabilities. Finally, we represent the kernel memory accesses in a bracketed format and employ a recursive algorithm to calculate the reuse distance profile. The results show that our approach can predict application characteristics accurately compared to another LLVM-based dynamic code analysis tool, Byfl.

Software Engineering,Performance,Programming Languages
Safe Memory-Leak Fixing for C Programs

Qing Gao,Yingfei Xiong,Yaqing Mi,Lu Zhang,Weikun Yang,Zhaoping Zhou,Bing Xie,Hong Mei

DOI: https://doi.org/10.1109/icse.2015.64

2015-01-01

Abstract:Automatic bug fixing has become a promising direction for reducing manual effort in debugging. However, general approaches to automatic bug fixing may face some fundamental difficulties. In this paper, we argue that automatic fixing of specific types of bugs can be a useful complement. This paper reports our first attempt towards automatically fixing memory leaks in C programs. Our approach generates only safe fixes, which are guaranteed not to interrupt normal execution of the program. To design such an approach, we have to deal with several challenging problems such as inter-procedural leaks, global variables, loops, and leaks from multiple allocations. We propose solutions to all the problems and integrate the solutions into a coherent approach. We implemented our inter-procedural memory leak fixing into a tool named Leak Fix and evaluated Leak Fix on 15 programs with 522k lines of code. Our evaluation shows that Leak Fix is able to successfully fix a substantial number of memory leaks, and Leak Fix is scalable for large applications.
Detecting Integer Bugs Based on Static and Dynamic Program Analysis

Ping CHEN,Hao HAN,Xiao-bing SHEN,Xin-chun YIN,Bing MAO,Li XIE

2010-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:In recent years, Integer bugs have been rising sharply and become a potential threat as it is often hidden behind other bugs. In this paper, we propose a tool which can automatically detect Integer bugs. We implement the tool based on static and dynamic program analysis. In the static phase, the tool decompiles a binary and creates the suspect instruction set. In the dynamic phase, it monitors the instructions in the suspect set and generates the test cases to further detect which instructions are real Integer bugs. Our tool has two advantages. First, it provides more accurate and sufficient type information. Second, static analysis reduces the instructions which are monitored at runtime. Experimental results shows that our tool can efficiently detect the Integer bugs in several real-world programs. In addition, our tool has no false negatives and low false positives.
Simber: Eliminating Redundant Memory Bound Checks Via Statistical Inference

Hongfa Xue,Yurong Chen,Fan Yao,Yongbo Li,Tian Lan,Guru Venkataramani

DOI: https://doi.org/10.1007/978-3-319-58469-0_28

2017-01-01

Abstract:Unsafe memory accesses in programs written using popular programming languages like C and C++ have been among the leading causes of software vulnerability. Memory safety checkers, such as Soft bound, enforce memory spatial safety by checking if accesses to array elements are within the corresponding array bounds. However, such checks often result in high execution time overhead due to the cost of executing the instructions associated with the bound checks. To mitigate this problem, techniques to eliminate redundant bound checks are needed. In this paper, we propose a novel framework, SIMBER, to eliminate redundant memory bound checks via statistical inference. In contrast to the existing techniques that primarily rely on static code analysis, our solution leverages a simple, model-based inference to identify redundant bound checks based on runtime statistics from past program executions. We construct a knowledge base containing sufficient conditions using variables inside functions, which are then applied adaptively to avoid future redundant checks at a function-level granularity. Our experimental results on real world applications show that SIMBER achieves zero false positives. Also, our approach reduces the performance overhead by up to 86.94% over Softbound, and incurs a modest 1.7% code size increase on average to circumvent the redundant bound checks inserted by Softbound.
Summary-based method to improve memory leak analysis

LEI Da,ZENG Qing-kai

2011-01-01

Abstract:In order to analyze memory leaks property in the source code more accurately and efficiently,this paper modeled memory leak and used various channels to improve the existing method.First,it used function summary method to increase efficiency.Second,it modeled LIB function and checkd infeasible path to reduce false positives.Third,it classified memory leaks and analysed the case of heap memory escape by parameter.It implements SMD tool which achieves good experimental results.
Graph Neural Networks Based Memory Inefficiency Detection Using Selective Sampling

Pengcheng Li,Yixin Guo,Yingwei Luo,Xiaolin Wang,Zhenlin Wang,Xu Liu

DOI: https://doi.org/10.1109/sc41404.2022.00090

2022-01-01

Abstract:Production software of data centers oftentimes suffers from unnecessary memory inefficiencies caused by inappropriate use of data structures, conservative compiler optimizations, and so forth. Nevertheless, whole-program monitoring tools often incur incredibly high overhead due to fine-grained memory access instrumentation. Consequently, the fine-grained monitoring tools are not viable for long-running, large-scale data center applications due to strict latency criteria (e.g., service-level agreement or SLA). To this end, this work presents a novel learning-aided system, namely Puffin, to identify three kinds of unnecessary memory operations including dead stores, silent loads and silent stores, by applying gated graph neural networks onto fused static and dynamic program semantics with respect to relative positional embedding. To deploy the system in large-scale data centers, this work explores a sampling-based detection infrastructure with high efficacy and negligible overhead. We evaluate Puffin upon the well-known SPEC CPU 2017 benchmark suite for four compilation options. Experimental results show that the proposed method is able to capture the three kinds of memory inefficiencies with as high accuracy as 96% and a reduced checking overhead by $5.66\times$ over the state-of-the-art tool.
Fast and precise points-to analysis with incremental CFL-reachability summarisation: preliminary experience

Lei Shang,Yi Lu,Jingling Xue

DOI: https://doi.org/10.1145/2351676.2351720

2012-01-01

Abstract:We describe our preliminary experience in the design and implementation of a points-to analysis for Java, called EMU, that enables developers to perform pointer-related queries in programs undergoing constant changes in IDEs. EMU achieves fast response times by adopting a modular approach to incrementally updating method summaries upon small code changes: the points-to information in a method is summarised indirectly by CFL reachability rather than directly by points-to sets. Thus, the impact of a small code change made in a method is localised, requiring only its affected part to be re-summarised just to reflect the change. EMU achieves precision by being context-sensitive (for both method invocation and heap abstraction) and field-sensitive. Our evaluation shows that EMU can be promisingly deployed in IDEs where the changes are small.
SMA: Eliminate Memory Spatial Errors via Saturation Memory Access

Chen Dongwei,Xu Daliang,Tong Dong,Sun Kang,Guan Xuetao,Yang Chun,Cheng Xu

2020-01-01

Abstract: Memory spatial error, i.e., buffer overflow, has been a well-known issue in computer security for a long time and remains one of the root causes of exploitable vulnerabilities. Existing tools focus on the detection of memory spatial errors and prevent intrusion by terminating the execution of the victim program. However, such tools cannot eliminate the vulnerabilities without patching the program. Unfortunately, in the increasingly popular embedded environment, deploying patches becomes harder because of the enormous number of devices. The limited resource in the embedded environment also prevents many existing tools to be used in the real world. This paper proposes the Saturation Memory Access (SMA), a memory spatial error elimination tool that prevents out-of-bound access without terminating the execution of a program. We use the tagged pointer scheme to store the boundary metadata of a memory object in the pointer itself, and correct the address to the object boundary upon detecting out-of-bound access. This method is based on a key observation that developers generally do not rely on out-of-bounds access to implement the program logic, so the correction of the address does not interfere with the execution of a program. We have implemented the prototype of SMA on LLVM 4.0.1 with two pointer encoding schemes designed for different tradeoff decisions between performance and memory usage. Experiments show that our prototype can stop nearly all attack forms in the RIPE benchmark and incurs 64\%-84\% overhead on SPEC CPU2017.
Realization of Dynamic Memory Leak Detection Technology under Linux

LIU Ming,RONG Meng-tian,GUO Fen,CHENG Man-li

DOI: https://doi.org/10.3969/j.issn.1009-2552.2007.09.009

2007-01-01

Abstract:Memory leak is a common problem in C/C+ + programs.Some leaks are hard to be found and takes a lot of time for the developer to debug and fix.This paper describes the implementation of a tool which can detect memory leaks for a program at runtime.With this tool,a program which is already loaded and running can be monitored at runtime.There is no need to do any modification to the program.It is useful for programs running for long-time and cannot be interrupted.
Scalable and Extensible Static Memory Safety Analysis with Summary over Access Path

Guang Chen,Min Zhou,Jiaguang Sun,Xiaoyu Song

DOI: https://doi.org/10.1109/APSEC.2018.00044

2018-01-01

Abstract:Static analysis is an effective way of checking memory safety issues in program. Usually, multiple analysis algorithms usually run together to achieve a precise analysis result. In this paper, a novel analysis frame work over access path is presented for incorporating analysis algorithms. A pointer analysis based on access path works as a base layer, alias and pointer information are automatically handled. An summary based checking algorithm is designed for checking real world project. Moreover, the framework is fully extensible and various analysis can be added as plugins. Experimental results show that our method has good precision on Juliet Test Suite and scales to large software.

A Tool for Estimating Memory Usage

Stack Bound Inference for Abstract Java Bytecode.

A Dynamic Approach to Detecting, Eliminating and Fixing Memory Leaks

Benchmark of Tools for Memory Leak

Spindle: Informed Memory Access Monitoring

TsmartGP: a tool for finding memory defects with pointer analysis

Who Ate My Memory? Towards Attribution in Memory Management

SCAD: Controlled Memory Allocation Analysis and Detection

Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems

A Source-Level Instrumentation Framework for the Dynamic Analysis of Memory Safety

A Lightweight Hybrid Hardware/software Approach for Object-Relative Memory Profiling

LLVM Static Analysis for Program Characterization and Memory Reuse Profile Estimation

Safe Memory-Leak Fixing for C Programs

Detecting Integer Bugs Based on Static and Dynamic Program Analysis

Simber: Eliminating Redundant Memory Bound Checks Via Statistical Inference

Summary-based method to improve memory leak analysis

Graph Neural Networks Based Memory Inefficiency Detection Using Selective Sampling

Fast and precise points-to analysis with incremental CFL-reachability summarisation: preliminary experience

SMA: Eliminate Memory Spatial Errors via Saturation Memory Access

Realization of Dynamic Memory Leak Detection Technology under Linux

Scalable and Extensible Static Memory Safety Analysis with Summary over Access Path