Abstract:As software systems continuously grow in size and complexity, performance and load related issues have become more common than functional issues. Load testing is usually performed before software releases to ensure that the software system can still provide quality service under a certain load. Therefore, one of the common challenges of load testing is to design realistic workloads that can represent the actual workload in the field. In particular, one of the most widely adopted and intuitive approaches is to directly replay the field workloads in the load testing environment. However, replaying a lengthy, e.g., 48 hours, field workloads is rather resource- and time-consuming, and sometimes even infeasible for large-scale software systems that adopt a rapid release cycle. On the other hand, replaying a short duration of the field workloads may still result in unrealistic load testing. In this work, we propose an automated approach to reduce the length of load testing that is driven by replaying the field workloads. The intuition of our approach is: if the measured performance associated with a particular system behaviour is already stable, we can skip subsequent testing of this system behaviour to reduce the length of the field workloads. In particular, our approach first clusters execution logs that are generated during the system runtime to identify similar system behaviours during the field workloads. Then, we use statistical methods to determine whether the measured performance associated with a system behaviour has been stable. We evaluate our approach on three open-source projects (i.e., OpenMRS, TeaStore, and Apache James). The results show that our approach can significantly reduce the length of field workloads while the workloads-after-reduction produced by our approach are representative of the original set of workloads. More importantly, the load testing results obtained by replaying the workloads after the reduction have high correlation and similar trend with the original set of workloads. Practitioners can leverage our approach to perform realistic field-replay based load testing while saving the needed resources and time. Our approach sheds light on future research that aims to reduce the cost of load testing for large-scale software systems.

Research of Load Testing and Result Based on Loadrunner

Model Based Load Testing of Web Applications

Theory and practice of load modeling in power systems

Load Modeling for Wide Area Power System

Workload Characterization of E-Commerce Site by Modeling Customer Behavior

Distance Based Root Cause Analysis and Change Impact Analysis of Performance Regressions

LTF: A Model-Based Load Testing Framework for Web Applications

Web Load Test Method Based on Log Analysis

Development of a network platform for remote hybrid dynamic testing

Reinforcement Learning Assisted Load Test Generation for E-Commerce Applications

Software performance testing based on workload characterization

Design and Implementation of Load Test for 3G Online Charging System

Norma: Towards Practical Network Load Testing

Towards virtualized and automated software performance test architecture

Reducing the Length of Field-Replay Based Load Testing

Development of Load System for High-Speed Railway Rolling

High-Volume Performance Test Framework using Big Data

Balanced Job Bounds Calculation for Approximating the E-Commerce Development Platform for 3-Tiered Web Applications: Performance Factors

An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform

An Experience Report On Applying Software Testing Academic Results In Industry: We Need Usable Automated Test Generation

Design of a loading plan format for an expert cargo loading system