Abstract:In this paper, we introduce Dreamweaver, which belongs to a new class of auto-regressive decision-making models known as large reasoning models (LRMs). Dreamweaver is designed to improve 3D floorplanning in electronic design automation (EDA) via an architecture that melds advancements in sequence-to-sequence reinforcement learning algorithms. A significant advantage of our approach is its ability to effectively reason over large discrete action spaces, which is essential for handling the numerous potential positions for various functional blocks in floorplanning. Additionally, Dreamweaver demonstrates strong performance even when trained on entirely random trajectories, showcasing its capacity to leverage sub-optimal or non-expert trajectories to enhance its results. This innovative approach contributes to streamlining the integrated circuit (IC) design flow and reducing the high computational costs typically associated with floorplanning. We evaluate its performance against a current state-of-the-art method, highlighting notable improvements.

What problem does this paper attempt to address?

The paper aims to address the problem of 3D floorplanning in the field of Electronic Design Automation (EDA). Specifically, the research team proposes a new model called Dreamweaver, which belongs to a type of Large Reasoning Models (LRMs). This is an autoregressive decision model optimized for handling Large Discrete Action Spaces (LDAS). The main contributions of the paper can be summarized as follows: 1. **Proposing the Dreamweaver model**: Dreamweaver is an autoregressive decision model based on a sequence-to-sequence reinforcement learning algorithm. It can effectively handle the numerous possible combinations of functional module placements in 3D floorplanning. By combining continuous action representation with k-nearest neighbor search techniques, Dreamweaver ensures the validity of the selected actions while maintaining sample efficiency. 2. **Addressing the large discrete action space problem**: The 3D floorplanning task involves placing various functional modules in an integrated circuit to optimize design criteria such as wire length, congestion, and thermal performance. Since each module can occupy multiple potential positions, this leads to a combinatorial explosion of possible configurations. Dreamweaver addresses this issue by leveraging the structural properties of action similarity, enabling efficient learning even in large action spaces. 3. **Learning from random trajectories**: Unlike traditional methods that require expert trajectory data for training, Dreamweaver can be trained on completely randomly generated trajectories and learn effective strategies from them. This approach not only reduces the cost of acquiring expert knowledge but also enhances the model's generalization capability. 4. **Multi-objective optimization**: In addition to traditional wire length optimization, Dreamweaver also considers other design constraints such as thermal performance and congestion estimation, achieving multi-objective optimization. 5. **Experimental results**: The paper validates Dreamweaver's significant improvements in reducing wire length compared to current state-of-the-art methods (e.g., Chipformer) through experiments on the MCNC benchmark dataset. Additionally, the experiments demonstrate the high accuracy of Dreamweaver's critic in predicting the returns of the remaining trajectory. In summary, this research proposes a novel and efficient solution to tackle the challenges in 3D floorplanning, making breakthrough progress particularly in handling large discrete action spaces.

Large Reasoning Models for 3D Floorplanning in EDA: Learning from Imperfections

GoodFloorplan: Graph Convolutional Network and Reinforcement Learning-Based Floorplanning

Web3D-based Automatic Furniture Layout System Using Recursive Case-Based Reasoning and Floor Field.

Floorplanning with Edge-Aware Graph Attention Network and Hindsight Experience Replay

LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation

Reimagining space layout design through deep reinforcement learning

Effective Analog ICs Floorplanning with Relational Graph Neural Networks and Reinforcement Learning

Spatial Geometric Reasoning for Room Layout Estimation via Deep Reinforcement Learning

Reinforcement learning for block decomposition of planar CAD models

Layout-aware Dreamer for Embodied Referring Expression Grounding

Learning Placement Order for Constructive Floorplanning

SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning

Deliberate Reasoning for LLMs as Structure-aware Planning with Accurate World Model

Making Large Language Models Better Planners with Reasoning-Decision Alignment

Deep Reinforcement Learning-based Large-scale Robot Exploration

World Models with Hints of Large Language Models for Goal Achieving

Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing

EDA-ML: Graph Representation Learning Framework for Digital IC Design Automation

Beyond LLMs: Advancing the Landscape of Complex Reasoning

Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models

SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Task Planning