Abstract:As a new area of machine learning research, the deep learning algorithm has attracted a lot of attention from the research community. It may bring human beings to a higher cognitive level of data. Its unsupervised pre-training step allows us to find high-dimensional representations or abstract features which work much better than the principal component analysis (PCA) method. However, it will face problems when being applied to deal with large scale data due to its intensive computation from many levels of training process against large scale data. The sequential deep learning algorithms usually can not finish the computation in an acceptable time. In this paper, we propose a many-core algorithm which is based on a parallel method and is used in the Intel Xeon Phi many-core systems to speed up the unsupervised training process of Sparse Autoencoder and Restricted Boltzmann Machine (RBM). Using the sequential training algorithm as a baseline to compare, we adopted several optimization methods to parallelize the algorithm. The experimental results show that our fully-optimized algorithm gains more than 300-fold speedup on parallelized Sparse Autoencoder compared with the original sequential algorithm on the Intel Xeon Phi coprocessor. Also, we ran the fully-optimized code on both the Intel Xeon Phi coprocessor and an expensive Intel Xeon CPU. Our method on the Intel Xeon Phi coprocessor is 7 to 10 times faster than the Intel Xeon CPU for this application. In addition to this, we compared our fully-optimized code on the Intel Xeon Phi with a Matlab code running on single Intel Xeon CPU. Our method on the Intel Xeon Phi runs 16 times faster than the Matlab implementation. The result also suggests that the Intel Xeon Phi can offer an efficient but more general-purposed way to parallelize the deep learning algorithm compared to GPU. It also achieves faster speed with better parallelism than the Intel Xeon CPU.

Discovery of biological networks using an optimized partial correlation coefficient with information theory algorithm on Stampede's Xeon and Xeon Phi processors

Parallel Accelerated Custom Correlation Coefficient Calculations for Genomics Applications

Fastgcn: A Gpu Accelerated Tool For Fast Gene Co-Expression Networks

Parallelized Kendall's Tau Coefficient Computation via SIMD Vectorized Sorting On Many-Integrated-Core Processors

Combining partial correlation and an information theory approach to the reversed engineering of gene co-expression networks

High Performance Computing of Gene Regulatory Networks using a Message-Passing Model

Using Matrix of Thresholding Partial Correlation Coefficients to Infer Regulatory Network

PCIT: an R package for weighted gene co-expression networks based on partial correlation and information theory approaches

Porting of the DBCSR library for Sparse Matrix-Matrix Multiplications to Intel Xeon Phi systems

MILC Code Performance on High End CPU and GPU Supercomputer Clusters

Training Large Scale Deep Neural Networks on the Intel Xeon Phi Many-Core Coprocessor

Asynchronous Parallel Dijkstra's Algorithm on Intel Xeon Phi Processor - How to Accelerate Irregular Memory Access Algorithm.

A high-performance computing toolset for relatedness and principal component analysis of SNP data

Enhanced OpenMP Algorithm to Compute All-Pairs Shortest Path on x86 Architectures

Parallelization of Bayesian Network Based SNPs Pattern Analysis and Performance Characterization on SMP/HT

NetSci: A Library for High Performance Biomolecular Simulation Network Analysis Computation

Parameterized Algorithms for Clustering PPI Networks

An efficient parallel block coordinate descent algorithm for large-scale precision matrix estimation using graphics processing units

Asynchronous Parallel Dijkstra’s Algorithm on Intel Xeon Phi Processor

Towards High-Performance Pattern Matching on Multi-Core Network Processing Platforms

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture