Abstract:Domain-Specific Languages (DSLs) provide the optimum balance between generalization and specialization that is crucial to getting the best performance for a particular domain. DSLs like Halide and GraphIt and their rich scheduling languages allow users to generate an implementation best suited for the algorithm and input. DSLs also provide the right abstraction for generating code for diverse architectures like GPUs, CPUs, and hardware accelerators. DSL compilers are massive, typically spanning tens of thousands of lines of code and need a frontend, some analysis and transformation passes, and target-specific code generation. These implementations usually require a great deal of compiler knowledge and domain experts cannot prototype DSLs without getting compiler experts involved. Using multi-stage programming in a high-level language like Scala, OCaml, or C++, is a great solution because it provides easy-to-use frontend and automatic code generation abilities. The DSL writers typically implement their abstraction as a library in the multi-stage programming language and use it to generate specialized code by providing partial inputs. This solves the problem only partially because DSLs like GraphIt have shown that several domain-specific analyses and transformations need to be performed to get the best performance. Special care has to be taken when targeting massively parallel architectures like GPUs where factors like load balancing, warp divergence, coalesced memory accesses play a critical role. In this paper, we demonstrate how to build an end-to-end DSL compiler framework and a graph DSL using multi-stage programming in C++. We show how the staged types can be extended to perform domain-specific data flow and control flow analyses and transformations. We also show how our generated CUDA code matches the performance of the code generated from the state-of-the-art graph DSL, GraphIt. We achieve all this in a very small fraction (8.4%) of the code size required to implement the traditional DSL compiler.

CONCORD: Towards a DSL for Configurable Graph Code Representation

CogDL: An Extensive Toolkit for Deep Learning on Graphs

Learning to Represent Programs with Graphs

deGraphCS : Embedding Variable-based Flow Graph for Neural Code Search

Learning the Implicit Semantic Representation on Graph-Structured Data

Unleashing the Power of Transformer for Graphs

DeepDSL: A Compilation-based Domain-Specific Language for Deep Learning

Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back

CodeGraph: Enhancing Graph Reasoning of LLMs with Code

Deep Learning with Dynamic Computation Graphs

CogDL: A Comprehensive Library for Graph Deep Learning

GraphIt to CUDA Compiler in 2021 LOC: A Case for High-Performance DSL Implementation via Staging with BuilDSL

DeepGD: A Deep Learning Framework for Graph Drawing Using GNN

Parallel Neurosymbolic Integration with Concordia

Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architectures

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases

Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning

Neuron to Graph: Interpreting Language Model Neurons at Scale

SIGN: Scalable Inception Graph Neural Networks

oneDNN Graph Compiler: A Hybrid Approach for High-Performance Deep Learning Compilation

Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks