Abstract:With the rapid growth of data and computing power, deep learning based approaches have become the main solution for many artificial intelligence problems such as image classification, speech recognition and computer vision. Several excellent deep learning (DL) frameworks including Tensorflow, MxNet and PyTorch have been made open-sourced, further accelerating the advance of the community. However, existing DL frameworks are not designed for applications involving high-dimensional sparse data, which exists widely in many successful online businesses such as search engine, recommender systems and online advertising. In these industrial scenarios, deep models are typically trained on large scale datasets with up to billions of sparse features and hundreds of billions of samples, bringing great challenges to DL framework. In this paper, we introduce a high-performance, large-scale and distributed DL framework named XDL which provides an elegant solution to fill the gap between general design of existing DL frameworks and industrial requirements arising from high-dimensional sparse data. Since 2016, XDL has been successfully deployed in Alibaba, serving many productions such as online advertising and recommender system. Running on hundreds of GPU cards in parallel, XDL can train deep models with tens of billions parameters within only several hours. Besides its excellent performance and flexibility, XDL is also friendly to developers. Algorithm scientists in Alibaba can develop and deploy new deep models with only several lines of simple codes. The XDL API and a reference implementation were released as an open-source package under the Apache 2.0 license in December, 2018 and are available at https://github.com/alibaba/xdeeplearning.

XDL: an industrial deep learning framework for high-dimensional sparse data

Deep Learning for Industrial Image: Challenges, Methods for Enriching the Sample Space and Restricting the Hypothesis Space, and Possible Issue

BigDL: A Distributed Deep Learning Framework for Big Data

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

BigDL 2.0: Seamless Scaling of AI Pipelines from Laptops to Distributed Cluster

An Orchestrated Empirical Study on Deep Learning Frameworks and Platforms

ElasticDL: A Kubernetes-native Deep Learning Framework with Fault-tolerance and Elastic Scheduling

An empirical study towards characterizing deep learning development and deployment across different frameworks and platforms

Efficient Deep Learning Board: Training Feedback Is Not All You Need

XFL: A High Performace, Lightweighted Federated Learning Framework

Optimizing Communication in Deep Reinforcement Learning with XingTian

FfDL : A Flexible Multi-tenant Deep Learning Platform

Design and implementation of DeepDSL: A DSL for deep learning

HPDL: Towards a General Framework for High-performance Distributed Deep Learning.

Model Asset eXchange: Path to Ubiquitous Deep Learning Deployment

AI Matrix: A Deep Learning Benchmark for Alibaba Data Centers

DeepXplore: Automated Whitebox Testing of Deep Learning Systems

DLBench: a comprehensive experimental evaluation of deep learning frameworks

SingleCaffe: an Efficient Framework for Deep Learning on a Single Node

Various Frameworks and Libraries of Machine Learning and Deep Learning: A Survey