Abstract:In recent years, due to the emergence of Big Data (terabytes or petabytes) and Big Model (tens of billions of parameters), there has been an ever-increasing need of parallelizing machine learning (ML) algorithms in both academia and industry. Although there are some existing distributed computing systems, such as Hadoop and Spark, for parallelizing ML algorithms, they only provide synchronous and coarse-grained operators (e.g., Map, Reduce, and Join, etc.), which may hinder developers from implementing more efficient algorithms. This motivated us to design a universal distributed platform termed KunPeng , that combines both distributed systems and parallel optimization algorithms to deal with the complexities that arise from large-scale ML. Specifically, KunPeng not only encapsulates the characteristics of data/model parallelism, load balancing, model sync-up, sparse representation, industrial fault-tolerance, etc., but also provides easy-to-use interface to empower users to focus on the core ML logics. Empirical results on terabytes of real datasets with billions of samples and features demonstrate that, such a design brings compelling performance improvements on ML programs ranging from Follow-the-Regularized-Leader Proximal algorithm to Sparse Logistic Regression and Multiple Additive Regression Trees. Furthermore, KunPeng's encouraging performance is also shown for several real-world applications including the Alibaba's Double 11 Online Shopping Festival and Ant Financial's transaction risk estimation.

Research on Parallelized Sentiment Classification Algorithms

Distributed Affinity Propagation Clustering Based on MapReduce

Vhadoop: A Scalable Hadoop Virtual Cluster Platform for MapReduce-Based Parallel Machine Learning with Performance Consideration

Parallel Sentiment Polarity Classification Method with Substring Feature Reduction

Parallel spectral clustering algorithm

Parallel Text Categorization of Massive Text Based on Hadoop

Parallelization of Classification Algorithms Based on SparkR

KunPeng: Parameter Server Based Distributed Learning Systems and Its Applications in Alibaba and Ant Financial

Sentiment Classification for Chinese Reviews: a Comparison Between SVM and Semantic Approaches

Evaluating Large Graph Processing in MapReduce Based on Message Passing

Parallelization of Machine Learning Algorithms Respectively on Single Machine and Spark

Parallel Approach and Platform for Large-Scale WEB Data Extraction

An Empirical Study Of Massively Parallel Bayesian Networks Learning For Sentiment Extraction From Unstructured Text

The performance of MapReduce: an in-depth study

A Scalable, Lexicon Based Technique for Sentiment Analysis

Big Data Quality Prediction in the Process Industry: A Distributed Parallel Modeling Framework

Parallel Image Texture Feature Extraction Under Hadoop Cloud Platform

A Semantic++ MapReduce Parallel Programming Model.

Study of ELM Algorithm Parallelization Based on Spark

Performance and Energy Consumption of Parallel Machine Learning Algorithms

Network Public Opinion Monitoring Based on MapReduce