Abstract:The complexity of chip design has consistently grown, adhering to Moore’s law. In this paper, we examine a crucial step in integrated circuit design called chip macro placement. Traditionally, human experts are consulted to optimize placement for reduced power consumption, but this requires significant effort. Recently, machine learning-based methods have emerged to address this task, showing promising results. Our work aims to tackle key bottlenecks in current reinforcement learning (RL) based methods, which often suffer from sparse rewards, large state–action search spaces, and unstable training, resulting in suboptimal solutions. To address these issues, we propose a novel approach called Hierarchical Reinforcement Learning for Placement (HRLP). This method distinguishes itself in two ways. First, we introduce a hierarchical RL framework that learns human expert design patterns by assigning agent-specific placement sub-tasks (referred to as options in reinforcement learning) instead of primitive actions at each time step within an episode. Second, we generate dense rewards for each episode by calculating the difference in local wirelength and congestion. We conducted experiments on eight public chip design benchmarks, adaptec and bigblue series. Our model has an average improvement of 9.29 % and 14.25 % on Half Perimeter Wire Length (HPWL), the huge improvement shows that our method is capable of converging to a better sub-optimal than DeepPlace and providing a promising future for AI to aid the design of circuit placement.

A Novel Deep Reinforcement Learning Methodology for EMI Mitigation in Multiscenario Pinmap Design

A Novel Deep Reinforcement Learning Methodology for EMI Mitigation in Multi-Scenario Pinmap Design

Deep Reinforcement Learning-Based Pin Map Optimization of BGA Package for EMC

Efficient EMI Analysis and Self-Shielding Design Method for Pin Map of System-in-Package

Pin Assignment Optimization of BGA Packages for EMI Mitigation Using Deep Reinforcement Learning

Deep Reinforcement Learning-Based Ground-Via Placement Optimization for EMI Mitigation

EMI Radiation Prediction and Structure Optimization of Packages by Deep Learning

Physics-Assisted Deep Reinforcement Learning-Based TSV Array Design for Signal and Power Integrity Co-Optimization

Machine Learning for Complex Emi Prediction, Optimization and Localization

Pin-opt: Graph Representation Learning for Large-scale Pin Assignment Optimization of Microbumps considering Signal and Power Integrity

Hierarchical Attention-Based Machine Learning Model for Radiation Prediction of WB-BGA Package

Deep Reinforcement Learning Based on Location-Aware Imitation Environment for RIS-Aided Mmwave MIMO Systems

EMI Radiation Prediction of Packages Based on Convolution Neural Network

Automatic SPICE- Integrated Reinforcement Learning for Decap Optimization for EMI and Power Integrity

Learning to Floorplan Like Human Experts Via Reinforcement Learning

DeepTH: Chip Placement with Deep Reinforcement Learning Using a Three-Head Policy Network

Progressive hierarchical deep reinforcement learning for defect wafer test

Hierarchical Reinforcement Learning for Chip-Macro Placement in Integrated Circuit

Miracle: Multi-Action Reinforcement Learning-Based Chip Floorplanning Reasoner

Machine Learning-Driven Discovery of High-Performance MEMS Disk Resonator Gyroscope Structural Topologies

On Joint Learning for Solving Placement and Routing in Chip Design