Abstract:......................................................................................................................... 3 1 Introduction ............................................................................................................. 3 1.1 Application Background ............................................................................... 4 1.2 Performance Demands for Deep Learning ................................................... 4 1.3 Existing Parallel Frameworks of Deep Learning ......................................... 4 1.4 Chapter Organization ................................................................................... 5 2 Concepts and Categories of Deep Learning ............................................................ 5 2.1 Deep Learning ............................................................................................. 5 2.1.1 Artificial Neural Networks ................................................................ 5 2.1.2 Concept of Deep Learning ................................................................. 7 2.2 Mainstream Deep Learning Models ............................................................. 8 2.2.1 Autoencoders ..................................................................................... 8 2.2.2 Back Propagation ............................................................................... 9 2.2.3 Convolutional Neural Network ........................................................ 11 3 Parallel Optimization for Deep Learning .............................................................. 13 3.1 Convolutional Architecture for Fast Feature Embedding ......................... 13 3.1.1 Introduction ...................................................................................... 13 3.1.2 CUDA Programming ....................................................................... 14 3.1.3 Architecture of Caffe ....................................................................... 17 3.1.4 Parallel Implementation of Convolution in Caffe ............................ 18 3.2 DistBelief .................................................................................................. 20 3.2.1 Introduction of DistBelief ................................................................ 20 3.2.2 Downpour SGD ............................................................................... 20 3.2.4 Sandblaster L-BFGS ........................................................................ 21 3.3 Deep Learning Based-on Multi-GPUs ...................................................... 22 3.3.1 Data Parallelism ............................................................................... 22 3.3.2 Model Parallelism ............................................................................ 23 3.3.3 Data-Model Parallelism ................................................................... 24 3.3.4 Example System of Multi-GPUs ..................................................... 25 4 Discussions ........................................................................................................... 26 4.1 Grand Challenges of Deep Learning with Big Data ................................. 26 4.1.1 Massive Amounts of Training Sample ............................................ 26 4.1.2 Incremental Streaming Data ............................................................ 26 4.1.3 Learning Speed with Big Data ......................................................... 26 4.1.4 Scalability of Deep Models .............................................................. 27 4.2 Future Work .............................................................................................. 27 References .................................................................................................................... 28 Deep Learning and Its Parallelization: Concepts and Instances Xiaqing Li, Guangyan Zhang, Keqin Li, and Weimin Zheng Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

Bioinformatics Tools with Deep Learning Based on GPU.

Deep learning for computational biology

Deep Learning and Machine Learning with GPGPU and CUDA: Unlocking the Power of Parallel Computing

Deep learning in bioinformatics: introduction, application, and perspective in big data era

Deep learning in bioinformatics: Introduction, application, and perspective in the big data era

Optimization of GPU Memory Usage for Training Deep Neural Networks.

A Learning Algorithm for Bayesian Networks and Its Efficient Implementation on GPUs

Deep learning optimization for drug-target interaction prediction in COVID-19 using graphic processing unit

Graphics processing units in bioinformatics, computational biology and systems biology

Modern deep learning in bioinformatics

Deep learning in bioinformatics

DeepProf: Performance Analysis for Deep Learning Applications via Mining GPU Execution Patterns

Review of the Applications of Deep Learning in Bioinformatics

Recent Advances of Deep Learning in Bioinformatics and Computational Biology

Deep Learning for Genomics: From Early Neural Nets to Modern Large Language Models

Deep Learning In Genomic And Medical Image Data Analysis: Challenges And Approaches

Deep learning in bioinformatics and biomedicine

A GPU-Based Training of BP Neural Network for Healthcare Data Analysis

Parallel Inference for Latent Dirichlet Allocation on Graphics Processing Units.

Deep Learning and Its Parallelization

A GPU-based computational framework that bridges neuron simulation and artificial intelligence