Abstract:This survey article reviews the challenges associated with deploying and optimizing big data applications and machine learning algorithms in cloud data centers and networks. The MapReduce programming model and its widely-used open-source platform; Hadoop, are enabling the development of a large number of cloud-based services and big data applications. MapReduce and Hadoop thus introduce innovative, efficient, and accelerated intensive computations and analytics. These services usually utilize commodity clusters within geographically-distributed data centers and provide cost-effective and elastic solutions. However, the increasing traffic between and within the data centers that migrate, store, and process big data, is becoming a bottleneck that calls for enhanced infrastructures capable of reducing the congestion and power consumption. Moreover, enterprises with multiple tenants requesting various big data services are challenged by the need to optimize leasing their resources at reduced running costs and power consumption while avoiding under or over utilization. In this survey, we present a summary of the characteristics of various big data programming models and applications and provide a review of cloud computing infrastructures, and related technologies such as virtualization, and software-defined networking that increasingly support big data systems. Moreover, we provide a brief review of data centers topologies, routing protocols, and traffic characteristics, and emphasize the implications of big data on such cloud data centers and their supporting networks. Wide ranging efforts were devoted to optimize systems that handle big data in terms of various applications performance metrics and/or infrastructure energy efficiency. Finally, some insights and future research directions are provided.

Optimizing the Topologies of Virtual Networks for Cloud-Based Big Data Processing

Optimal Construction of Virtual Networks for Cloud-based MapReduce Workflows.

Research on Virtual Network Design Based on Virtual Router

Managing Recurrent Virtual Network Updates in Multi-Tenant Datacenters: A System Perspective

Sdn Based Vxlan Optimization In Cloud Computing Networks

A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks

Distance-aware Virtual Cluster Performance Optimization: A Hadoop Case Study

Optimizing Distributed Networking with Big Data Scheduling and Cloud Computing

Vhadoop: A Scalable Hadoop Virtual Cluster Platform for MapReduce-Based Parallel Machine Learning with Performance Consideration

Virtual Machine Based Energy-Efficient Data Center Architecture for Cloud Computing: A Performance Perspective.

A Joint Optimization of Operational Cost and Performance Interference in Cloud Data Centers

Improving performance by network-aware virtual machine clustering and consolidation

The performance of MapReduce: an in-depth study

Scalability Analysis and Improvement of Hadoop Virtual Cluster with Cost Consideration

Topology-Aware Resource Allocation for IoT Services in Clouds

Virtual Network Deployment in Hybrid Data Center Networks.

Energy Efficient Virtual Machine Placement with an Improved Ant Colony Optimization over Data Center Networks

Optimal VM Placement for Traffic Scalability Using Markov Chain in Cloud Data Centre Networks

Performance optimization of computing task scheduling based on the Hadoop big data platform

Joint Study On Optimizations Of Data Center Deployment, Vm Assignment And Migration

Network-Aware Re-Scheduling: Towards Improving Network Performance of Virtual Machines in a Data Center.