Abstract:Coping with the growing demand for data and parameters in complex neural network (NN) models of contemporary times typically involves distributing them across multiple devices, giving rise to the device placement problem. Several widely adopted solutions to this challenge leverage graph embedding schemes. However, the current graph embedding models employed for device placement suffer from several limitations, such as being highly time-consuming in the search for optimal parallelization strategies and yielding suboptimal parallel results. In this paper, we develop a novel graph embedding approach CP-GNNAK for efficient device placement. CP-GNNAK mainly depends on an efficient subgraph-based node embedding strategy, namely GNN as kernel, and a novel cosine phase position embedding scheme. CP-GNNAK goes beyond encoding only the immediate neighbors by deriving the representation of each node from the encoding of a surrounding subgraph. This expanded approach significantly enhances the efficiency of device placement through the integration of subgraph sampling strategies. Additionally, the cosine phase position embedding not only considers the positional information between nodes but also captures the directional information underlying the nodes. This adds richness to the node embeddings and improves the performance of device placement. The experimental results clearly indicate that the CP-GNNAK model achieves the state-of-the-art device placement results. Specifically, the experimental results reveal a remarkable 13.77% improvement in runtime when employing CP-GNNAK compared to Placeto. CP-GNNAK also outperforms GraphSAGE by 7.36% and P-GNN by 0.94%. Turning to computation time, CP-GNNAK exhibits a substantial speedup of 3.40 × over Placeto, 3.87 × over GraphSAGE, and 3.20 × over P-GNN.

Parallel placement algorithm based on neural network

An Efficient Algorithm for Mapping Deep Learning Applications on the NoC Architecture

THE PARALLEL PROCESSING METHOD IN NEURAL NETWORKS FOR TRAFFIC EQUILIBRIUM ASSIGNMENT PROBLEMS

A novel device placement approach based on position-aware subgraph neural networks

System partitioning on MCM using a new neural network model

Implementation of Global Placement Based on FastPlace Algorithm

Core Placement Optimization for Multi-chip Many-core Neural Network Systems with Reinforcement Learning.

Parallelization Techniques for Verifying Neural Networks

Parallel Global Placement on CPU Via Parallel Reduction

Indoor Positioning Algorithm Based on Parallel Multilayer Neural Network

Aware: Adaptive Distributed Training with Computation, Communication and Position Awareness for Deep Learning Model.

Optimize Resource Placement for In-Network Computing

Dynamic Universal Approximation Theory: Foundations for Parallelism in Neural Networks

Network I/O Load Based Virtual Machine Placement Algorithm in HPC Cloud

A General Neural Computer Architecture and Its Ann-Based Task Assignment Method for Parallel Distributed Processing

Nnscaler: Constraint-Guided Parallelization Plan Generation for Deep Learning Training.

A Sa-Based Parallel Method For Fpga Placement

Parallel Algorithms Align with Neural Execution

Parallel Link Prediction in Complex Network Using MapReduce

A Theoretical Framework for Parallel Implementation of Deep Higher Order Neural Networks

Simulated Annealing and Parallel Realization: Application on VLSI Block Placement