Evaluating Ethereum Reentrancy Detection Tools Via Mutation Testing

Kaitai Zhu,Xingya Wang,Zhenyu Chen,Song Huang,Junhua Wu
DOI: https://doi.org/10.1109/issre59848.2023.00067
2023-01-01
Abstract:Reentrancy vulnerabilities in Ethereum smart contracts have caused huge financial losses in recent years, promoting the development of reentrancy detection tools. Evaluation of reentrancy detection tools has emerged as an essential research focus consequently. However, despite numerous reentrancy detection evaluation approaches, there is no systematically classified set of reentrancy vulnerabilities in evaluation. This oversight may result in a limited understanding of the actual effectiveness of reentrancy detection tools. This paper employs mutation testing to generate classified sets of reentrancy vulnerabilities and evaluate reentrancy detection tools. To determine the types of reentrancy that can be mutated, we use symbolic execution to identify potential reentrancy paths in smart contracts. For each reentrancy type, we design one to three mutation operators. Then, we employ pattern matching to locate statements where mutation operators can be applied, thus generating sets of reentrancy vulnerabilities of different types. We implemented and compared our mutation testing tool with other tools. Results show that we generated at least 29% more classified reentrancy mutants. Furthermore, we evaluated two state-of-the-art reentrancy detection tools based on the classified reentrancy mutants. The findings provide directions for advancing reentrancy detection tools.
What problem does this paper attempt to address?