Abstract:This survey article reviews the challenges associated with deploying and optimizing big data applications and machine learning algorithms in cloud data centers and networks. The MapReduce programming model and its widely-used open-source platform; Hadoop, are enabling the development of a large number of cloud-based services and big data applications. MapReduce and Hadoop thus introduce innovative, efficient, and accelerated intensive computations and analytics. These services usually utilize commodity clusters within geographically-distributed data centers and provide cost-effective and elastic solutions. However, the increasing traffic between and within the data centers that migrate, store, and process big data, is becoming a bottleneck that calls for enhanced infrastructures capable of reducing the congestion and power consumption. Moreover, enterprises with multiple tenants requesting various big data services are challenged by the need to optimize leasing their resources at reduced running costs and power consumption while avoiding under or over utilization. In this survey, we present a summary of the characteristics of various big data programming models and applications and provide a review of cloud computing infrastructures, and related technologies such as virtualization, and software-defined networking that increasingly support big data systems. Moreover, we provide a brief review of data centers topologies, routing protocols, and traffic characteristics, and emphasize the implications of big data on such cloud data centers and their supporting networks. Wide ranging efforts were devoted to optimize systems that handle big data in terms of various applications performance metrics and/or infrastructure energy efficiency. Finally, some insights and future research directions are provided.

Studies on the Large Scale Data Processing Technologies Used in Servers for Cloud Computing

Visual Analysis of Cloud Computing Performance Using Behavioral Lines

An Efficient Massive Data Processing Model in the Cloud -- A Preliminary Report

The performance of MapReduce: an in-depth study

Real-Time Big Data Processing Framework: Challenges and Solutions

The Performance of MapReduce

DataCloud: an Efficient Massive Data Mining and Analysis Framework on Large Clusters.

Recent Developments in Parallel and Distributed Computing for Remotely Sensed Big Data Processing

Providing Scalable Database Services On The Cloud

Efficient B-tree Based Indexing for Cloud Data Processing.

Research on Data Storage Technology in Cloud Computing Environment

Big Data Storage Architecture Design in Cloud Computing

Scalable Data Partitioning Techniques for Distributed Data Processing in Cloud Environments: A Review

Implementation Issues of A Cloud Computing Platform.

Moving big data to the cloud

Building a Productive Domain-Specific Cloud for Big Data Processing and Analytics Service

Research on Service-Oriented Data Mining Engine Based on Cloud Computing

Transplantation of Data Mining Algorithms to Cloud Computing Platform when Dealing Big Data

A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks

Survey of Distributed Computing Frameworks for Supporting Big Data Analysis

Analyzing large-scale Data Cubes with user-defined algorithms: A cloud-native approach