TimeMachine: Time-travel Testing of Android Apps

Zhen Dong

2020-01-01

Abstract:We introduce a prototype of an automated Android app testing technique. The distinctive feature of the technique is that it saves program states in the execution and restores an interesting state previously saved for further exploration when progress is slow so as to continuously explore new program behavior (namely time-travel testing). Our empirical results show that our technique outperforms the-state-of-art techniques including Sapienz (used at Facebook). Our technique has been developed as an fully automated Android testing tool and packed into an Docker image for ease of use. Our prototype is publicly available at https://github.com/DroidTest/ TimeMachine. 1 TIME-TRAVEL TESTING Idea. Time-travel testing is a mobile app testing approach, which saves app states in the execution and restores an interesting state previously-saved for further exploration when progress is slow in the hope of exploring new program behavior efficiently. An interesting state is identified based on runtime observations, e.g., a state is interesting or has potential to trigger new program behavior if the state is visited for the first time. App state is identified by GUI layout, two screens have an identifical state (abstractly) if they have the same GUI layout. App states can be saved by taking system snapshots when running on an emulator. Boosting Mobile App Testing. Time-travel testing records interesting states observed, and can travel back to any previous state to launch a new execution. This enhances existing mobile app testing techniques to explore program behavior more efficiently. For instance, random testing (like Android Monkey) generates a very long sequence of events to exercise apps, but often suffers from low efficiency that execution loops in certain states (e.g., main screen). Time-travel testing allows random testing to jump out of such state loops and drive execution to infrequently visited states. Search-based testing techniques systematically evolve a population of event sequences so as to achieve certain objectives such as maximal code coverage. The hope is that the mutation of fit event sequences leads to the generation of even fitter sequences. However, the evolution of event sequences may be ineffective. Pertinent app states which contributed to the original sequence’s fitness may not be reached by a mutated event sequence because the original path through the state space is truncated at the point of mutation. Time-travel testing allows search-based techniques to identify the part of input events that leads to pertinent app states and avoid making any changes to this part during mutation, and generate RQ1:Effectiveness of Time-travel Strategy 42 44 46 48 50 52 54 56 Monkey TimeMachine Statement coverage (%)