Abstract:The Deep Learning (DL) community sees many novel topologies published each year. Achieving high performance on each new topology remains challenging, as each requires some level of manual effort. This issue is compounded by the proliferation of frameworks and hardware platforms. The current approach, which we call "direct optimization", requires deep changes within each framework to improve the training performance for each hardware backend (CPUs, GPUs, FPGAs, ASICs) and requires $\mathcal{O}(fp)$ effort; where $f$ is the number of frameworks and $p$ is the number of platforms. While optimized kernels for deep-learning primitives are provided via libraries like Intel Math Kernel Library for Deep Neural Networks (MKL-DNN), there are several compiler-inspired ways in which performance can be further optimized. Building on our experience creating neon (a fast deep learning library on GPUs), we developed Intel nGraph, a soon to be open-sourced C++ library to simplify the realization of optimized deep learning performance across frameworks and hardware platforms. Initially-supported frameworks include TensorFlow, MXNet, and Intel neon framework. Initial backends are Intel Architecture CPUs (CPU), the Intel(R) Nervana Neural Network Processor(R) (NNP), and NVIDIA GPUs. Currently supported compiler optimizations include efficient memory management and data layout abstraction. In this paper, we describe our overall architecture and its core components. In the future, we envision extending nGraph API support to a wider range of frameworks, hardware (including FPGAs and ASICs), and compiler optimizations (training versus inference optimizations, multi-node and multi-device scaling via efficient sub-graph partitioning, and HW-specific compounding of operations).

Reinforced Genetic Algorithm Learning for Optimizing Computation Graphs

Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time

Using Graph Neural Networks to model the performance of Deep Neural Networks

Learning Heuristics over Large Graphs via Deep Reinforcement Learning

Optimizing DNN computation graph using graph substitutions

oneDNN Graph Compiler: A Hybrid Approach for High-Performance Deep Learning Compilation

Optimizing DNN Computation with Relaxed Graph Substitutions

GSO:A GNN-based Deep Learning Computation Graph Substitutions Optimization Framework

Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs

Optimizing Tensor Computation Graphs with Equality Saturation and Monte Carlo Tree Search

Deep Learning with Dynamic Computation Graphs

Deep Learning Compiler Load Balancing Optimization Method for Model Training

Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning

Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilation

AutoGraph: Optimizing DNN Computation Graph for Parallel GPU Kernel Execution.

Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs

Learning Execution through Neural Code Fusion

ALT: Breaking the Wall Between Data Layout and Loop Optimizations for Deep Learning Compilation

Compute-Efficient Neural Network Architecture Optimization by a Genetic Algorithm

Efficient Learning of Discrete-Continuous Computation Graphs

Reinforcement Learning for Scalable Logic Optimization with Graph Neural Networks