Abstract:Deep Learning-based (DL) applications are becoming increasingly popular and advancing at an unprecedented pace. While many research works are being undertaken to enhance Deep Neural Networks (DNN) -- the centerpiece of DL applications -- practical deployment challenges of these applications in the Cloud and Edge systems, and their impact on the usability of the applications have not been sufficiently investigated. In particular, the impact of deploying different virtualization platforms, offered by the Cloud and Edge, on the usability of DL applications (in terms of the End-to-End (E2E) inference time) has remained an open question. Importantly, resource elasticity (by means of scale-up), CPU pinning, and processor type (CPU vs GPU) configurations have shown to be influential on the virtualization overhead. Accordingly, the goal of this research is to study the impact of these potentially decisive deployment options on the E2E performance, thus, usability of the DL applications. To that end, we measure the impact of four popular execution platforms (namely, bare-metal, virtual machine (VM), container, and container in VM) on the E2E inference time of four types of DL applications, upon changing processor configuration (scale-up, CPU pinning) and processor types. This study reveals a set of interesting and sometimes counter-intuitive findings that can be used as best practices by Cloud solution architects to efficiently deploy DL applications in various systems. The notable finding is that the solution architects must be aware of the DL application characteristics, particularly, their pre- and post-processing requirements, to be able to optimally choose and configure an execution platform, determine the use of GPU, and decide the efficient scale-up range.

Examining the Impact of Platform Properties on Quality Attributes

Quality Attributes on Quantum Computing Platforms

Quantum Computing Platforms: Assessing the Impact on Quality Attributes and SDLC Activities

Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective.

A Survey of Virtualization Technologies With Performance Testing

High Performance and Security in Cloud Computing

Evaluating Services on the Cloud Using Ontology QoS Model

Closed or Open Platform? the Nature of Platform and a Qualitative Comparative Analysis of the Performance Effect of Platform Openness

Vtestkit: A Performance Benchmarking Framework for Virtualization Environments

Evaluate The Performance And Scalability Of Image Deployment In Virtual Data Center

Human-Centric Composite-Quality Modeling and Assessment for Virtual Desktop Clouds

On the Benefit of Virtualization: Strategies for Flexible Server Allocation

Exploring the Impact of Virtualization on the Usability of the Deep Learning Applications

Characterizing Workload of Web Applications on Virtualized Servers

Incidence of software quality attributes in the design, construction and deployment of Cloud architectural environments

An Empirical Study of Vmm Overhead, Configuration Performance, and Scalability

Experimental Assessment of Containers Running on Top of Virtual Machines

Performance, Fault-Tolerance and Scalability Analysis of Virtual Infrastructure Management System

Virt-B: Towards Performance Benchmarking of Virtual Machine Systems.

Cloud-Native Architectural Characteristics and their Impacts on Software Quality: A Validation Survey

Towards Architecture-Based Management of Platforms in the Cloud.