Abstract:GUI testing is significant in the SE community. Most existing frameworks are intrusive and only support some specific platforms. With the development of distinct scenarios, diverse embedded systems or customized operating systems on different devices do not support existing intrusive GUI testing frameworks. Some approaches adopt robotic arms to replace the interface invoking of mobile apps under test and use computer vision technologies to identify GUI elements. However, some challenges are unsolved. First, existing approaches assume that GUI screens are fixed so that they cannot be adapted to diverse systems with different screen conditions. Second, existing approaches use XY-plane robotic arms, which cannot flexibly simulate testing operations. Third, existing approaches ignore compatibility bugs and only focus on crash bugs. A more practical approach is required for the non-intrusive scenario. We propose a practical non-intrusive GUI testing framework with visual robotic arms. RoboTest integrates novel GUI screen and widget detection algorithms, adaptive to detecting screens of different sizes and then to extracting GUI widgets from the detected screens. Then, a set of testing operations is applied with a 4-DOF robotic arm, which effectively and flexibly simulates human testing operations. During app exploration, RoboTest integrates the Principle of Proximity-guided exploration strategy, choosing close widgets of the previous targets to reduce robotic arm movement overhead and improve exploration efficiency. RoboTest can effectively detect some compatibility bugs beyond crash bugs with a GUI comparison on different devices of the same test operations. We evaluate RoboTest with 20 mobile apps, with a case study on an embedded system. The results show that RoboTest can effectively, efficiently, and generally explore AUTs to find bugs and reduce exploration time overhead.

Augmented testing to support manual GUI-based regression testing: An empirical study

Machine Learning and Evolutionary Computing for GUI-based Regression Testing

Visual Testing of GUIs by Abstraction

Effective GUI-oriented Automated Testing Platform for Pervasive Computing Software Environment

An Ontology-Based Approach for GUI Testing

A Test Automation Solution On Gui Functional Test

A Novel Automated GUI Testing Echnology Based on Image Recognition

Estimating Return on Investment for GUI Test Automation Tools

AUITestAgent: Automatic Requirements Oriented GUI Function Testing

Practical Non-Intrusive GUI Exploration Testing with Visual-based Robotic Arms

Creating Gui Testing Tools Using Accessibility Technologies

GUI Testing Assisted by Human Knowledge: Random Vs. Functional.

Using a pilot study to derive a GUI model for automated testing

Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing

Maintaining and Evolving GUI-directed Test Scripts

Automating GUI-based Software Testing with GPT-3

Practical GUI Testing of Android Applications Via Model Abstraction and Refinement

Visual GUI testing in practice: An extended industrial case study

GUI Testing Arena: A Unified Benchmark for Advancing Autonomous GUI Testing Agent

The Approach of Graphical User Interface Testing Guided by Bayesian Model

Experimental Assessment of Manual Versus Tool-Based Maintenance of Gui-Directed Test Scripts