Abstract:Software systems are widely employed in society. With a limited amount of testing resource available, testing resource allocation among components of a software system becomes an important issue. Most existing research on the testing resource allocation problem takes a single-objective optimization approach, which may not adequately address all the concerns in the decision-making process. In this paper, an architecture-based multi-objective optimization approach to testing resource allocation is proposed. An architecture-based model is used for system reliability assessment, which has the advantage of explicitly considering system architecture over the reliability block diagram (RBD)-based models, and has good flexibility to different architectural alternatives and component changes. A system cost modeling approach which is based on well-developed software cost models is proposed, which would be a more flexible, suitable approach to the cost modeling of software than the approach adopted by others which is based on an empirical cost model. A multi-objective optimization model is developed for the testing resource allocation problem, in which the three major concerns in the testing resource allocation problem, i.e., system reliability, system cost, and the total amount of testing resource consumed, are taken into consideration. A multi-objective evolutionary algorithm (MOEA), called multi-objective differential evolution based on weighted normalized sum (WNS-MODE), is developed. Experimental studies are presented, and the experiments show several results. 1) The proposed architecture-based multi-objective optimization approach can identify the testing resource allocation strategy which has a good trade-off among optimization objectives. 2) The developed WNS-MODE is better than the MOEA developed in recent research, called HaD-MOEA, in terms of both solution quality and computational efficiency. 3) The WNS-MODE seems quite robust from the sensitivity analysis results.

Acceleration Testing Method of Safety-critical Software with Testing Resource Constraint

Enhancing Performance of Random Testing Through Markov Chain Monte Carlo Methods

Research on Fatigue Intensifying Test of Driving Axle and Estimating Confidence Interval of Fatigue Life

A Markov Decision Approach to Optimize Testing Profile in Software Testing

Cross-Entropy Method Based on Markov Decision Process for Optimal Software Testing

Software Reliability Test Methods Based on Component Composition and Markov Process

Estimation Method of Software Reliability for Safety-critical System

Optimal testing resource allocation, and sensitivity analysis in software development

A mixed model to evaluate random hardware failures of whole-redundancy system in ISO 26262 based on fault tree analysis and Markov chain

A grey incidence-based algorithm for optimization of accelerated degradation test

Reliability-based Optimization of Maintenance Scheduling of Mechanical Components with the Method of Accelerated Tests

Optimal design for step-stress accelerated degradation testing with competing failure modes

Dynamic Testing for Autonomous Vehicles Using Random Quasi Monte Carlo

Optimizing design method of operational profile in statistical testing

A Quantitative Approach of Generating Challenging Testing Scenarios Based on Functional Safety Standard

Quantitative Reliability Assessment for Safety Related Systems Using Markov Models

An Architecture-Based Multi-Objective Optimization Approach to Testing Resource Allocation

A Stochastic Combinatorial Optimization Model for Test Sequence Optimization.

Reliability estimation from two types of accelerated testing considering individual difference and measurement error

A Markov chain model for statistical software testing

Optimal designs of accelerated degradation tests with random shock failures and measurement errors