Abstract:The complexity of chip design has consistently grown, adhering to Moore’s law. In this paper, we examine a crucial step in integrated circuit design called chip macro placement. Traditionally, human experts are consulted to optimize placement for reduced power consumption, but this requires significant effort. Recently, machine learning-based methods have emerged to address this task, showing promising results. Our work aims to tackle key bottlenecks in current reinforcement learning (RL) based methods, which often suffer from sparse rewards, large state–action search spaces, and unstable training, resulting in suboptimal solutions. To address these issues, we propose a novel approach called Hierarchical Reinforcement Learning for Placement (HRLP). This method distinguishes itself in two ways. First, we introduce a hierarchical RL framework that learns human expert design patterns by assigning agent-specific placement sub-tasks (referred to as options in reinforcement learning) instead of primitive actions at each time step within an episode. Second, we generate dense rewards for each episode by calculating the difference in local wirelength and congestion. We conducted experiments on eight public chip design benchmarks, adaptec and bigblue series. Our model has an average improvement of 9.29 % and 14.25 % on Half Perimeter Wire Length (HPWL), the huge improvement shows that our method is capable of converging to a better sub-optimal than DeepPlace and providing a promising future for AI to aid the design of circuit placement.

FPGA Divide-and-Conquer Placement using Deep Reinforcement Learning

Improving Simulated Annealing Algorithm for FPGA Placement Based on Reinforcement Learning

Chip Placement with Deep Reinforcement Learning

GoodFloorplan: Graph Convolutional Network and Reinforcement Learning-Based Floorplanning

Learning Placement Order for Constructive Floorplanning

DeepTH: Chip Placement with Deep Reinforcement Learning Using a Three-Head Policy Network

Learning to Floorplan Like Human Experts Via Reinforcement Learning

Hierarchical Reinforcement Learning for Chip-Macro Placement in Integrated Circuit

DREAMPlaceFPGA: An Open-Source Analytical Placer for Large Scale Heterogeneous FPGAs using Deep-Learning Toolkit

On Joint Learning for Solving Placement and Routing in Chip Design

FPGA placement by using a genetic algorithm

RALP: Reconvergence-aware layer partitioning for 3D FPGAs

Imbalanced Large Graph Learning Framework for FPGA Logic Elements Packing Prediction

Routing and Placement of Macros using Deep Reinforcement Learning

High-performance Placement Engine for Modern Large-scale FPGAs With Heterogeneity and Clock Constraints

Chiplet Placement Order Exploration Based on Learning to Rank with Graph Representation

Non-Overlapping Placement of Macro Cells based on Reinforcement Learning in Chip Design

Miracle: Multi-Action Reinforcement Learning-Based Chip Floorplanning Reasoner

DrPlace: A Deep Learning Based Routability-Driven VLSI Placement Algorithm

Wirelength-Driven Fast Placement Algorithm for Island Style FPGAs