Abstract:Cloud Computing has been increasingly accepted as a promising computing paradigm in industry, with one of the most common delivery models being Infrastructure as a Service (IaaS). An increasing number of providers have started to supply public IaaS services with different terminologies, definitions, and goals. As such, understanding the full scope of performance evaluation of candidate services would be crucial and beneficial for both service customers (e.g., cost-benefit analysis) and providers (e.g., direction of improvement). Given the numerous and diverse IaaS service features to be evaluated, a natural strategy is to implement different types of evaluation experiments separately. Unfortunately, it could be hard to fairly distinguish between different experimental types due to different environments and techniques that may be adopted by different evaluators. To overcome such obstacles, we have first established a novel taxonomy to help profile and clarify the nature of IaaS services performance evaluation and then built a three-layer conceptual model to generalize the existing performance evaluation practices. Using relevant elements/classifiers in the taxonomy and conceptual model, evaluators can construct natural language-style descriptions and experimental design blueprints to outline the evaluation scope and also to guide new evaluation implementations. In essence, the generated descriptions and blueprints abstractly define and characterize the actual evaluation work. This enables relatively fair and rational comparisons between different performance evaluations according to their abstract characteristics.

A cloud-based TaaS infrastructure with tools for SaaS validation, performance and scalability evaluation

Testing as a Service (TaaS) on Clouds

Automated Model-Based Performance Testing for PaaS Cloud Services

[email protected]: the virtual test lab on the cloud

Testing As a Service over Cloud

SaaS performance and scalability evaluation in clouds

Vee@Cloud: the virtual test lab on the cloud

SaaS Testing on Clouds - Issues, Challenges and Needs

A Framework of Testing As a Service

Mobile Testing-as-a-Service (MTaaS) -- Infrastructures, Issues, Solutions and Needs

Cloud testing-issues, challenges, needs and practice

Cloud testing tools.

Research of CAPP Service Based on SaaS

Performing unit testing based on testing as a service (TaaS) approach

Cloud Scalability Measurement And Testing

Towards a Scalable and Robust Multi-Tenancy SaaS

Scalable Architectures for SaaS

Software-as-a-service (SaaS): perspectives and challenges

A Model of Cloud Based Application Environment for Software Testing

SAASQUAL: A Quality Model For Evaluating SaaS on The Cloud Computing Environment

On the Conceptualization of Performance Evaluation of Iaas Services