Abstract:In the field of very large-scale integration (VLSI) chip design, chip floorplanning plays a crucial role as it directly influences key optimization objectives such as placement wirelength. This, in turn, affects signal delay, power efficiency, routability, and overall cost. However, traditional reinforcement learning (RL) methods for chip floorplanning often oversimplify this complex and dynamic task. They tend to overlook the cascading effects of module placements and fail to fully comprehend the intricate interdependencies that are vital for making informed decisions. To address these challenges, we introduce an innovative approach that combines hypergraph graph convolutional networks (HyperGCN) with deep recurrent Q-networks (DRQN). This integration allows us to capture the nuanced dynamics and interconnected aspects of chip design more effectively. We enhance the traditional Markov decision process (MDP) model by incorporating a state characterization layer based on long short-term memory (LSTM) technology. Initially, HyperGCN efficiently encodes netlist information, simplifying complex graph structures into lower-dimensional vectors, thereby enhancing knowledge processing. Subsequently, we treat the chip as an agent and apply the DRQN algorithm to optimize the module layout. DRQN’s LSTM utilizes a recurrent layer structure to grasp dependencies between modules, combined with DQN’s optimization capabilities, enabling us to navigate the complexities of floorplanning. Our approach improves the state representation by encompassing a broader understanding of interconnected module characteristics. This allows our agents to make decisions that take into account the collective impact of module adjustments, rather than viewing each change in isolation. This comprehensive state representation, which includes diverse features and their evolving relationships, significantly enhances the decision-making capabilities of our agents. Our extensive experiments demonstrate that our method outperforms traditional heuristic-based and other learning-based floorplanning techniques. To the best of our knowledge, this is the first application of LSTM with DQN in circuit design, representing a significant advancement in the field.

LSTM-Characterized Approach for Chip Floorplanning: Leveraging HyperGCN and DRQN

GoodFloorplan: Graph Convolutional Network and Reinforcement Learning-Based Floorplanning

Floorplanning with Edge-Aware Graph Attention Network and Hindsight Experience Replay

GraphPlanner: Floorplanning with Graph Neural Network.

Spacecraft Attitude Maneuver Planning Based on Deep Reinforcement Learning under Complex Constraints

Learning Placement Order for Constructive Floorplanning

Chip Placement with Deep Reinforcement Learning

A graph placement methodology for fast chip design

Hierarchical Reinforcement Learning for Chip-Macro Placement in Integrated Circuit

RLPlanner: Reinforcement Learning based Floorplanning for Chiplets with Fast Thermal Analysis

Hierarchical Graph Learning-Based Floorplanning with Dirichlet Boundary Conditions

FLTRNN: Faithful Long-Horizon Task Planning for Robotics with Large Language Models

Effective Analog ICs Floorplanning with Relational Graph Neural Networks and Reinforcement Learning

Non-Overlapping Placement of Macro Cells based on Reinforcement Learning in Chip Design

On Joint Learning for Solving Placement and Routing in Chip Design

Solving non-permutation flow-shop scheduling problem via a novel deep reinforcement learning approach

ChiPFormer: Transferable Chip Placement via Offline Decision Transformer

Floorplanning with Graph Attention

Chiplet Placement Order Exploration Based on Learning to Rank with Graph Representation

A novel method based on deep reinforcement learning for machining process route planning

Multi-objective optimization in fixed-outline floorplanning with reinforcement learning