Abstract:In human cognition theory, human thinking is governed by two systems: the fast and intuitive System 1 and the slower but more deliberative System 2. Recent studies have shown that incorporating System 2 process into Transformers including large language models (LLMs), significantly enhances their reasoning capabilities. Nevertheless, models that purely resemble System 2 thinking require substantially higher computational costs and are much slower to respond. To address this challenge, we present Dualformer, a single Transformer model that seamlessly integrates both the fast and slow reasoning modes. Dualformer is obtained by training on data with randomized reasoning traces, where different parts of the traces are dropped during training. The dropping strategies are specifically tailored according to the trace structure, analogous to analyzing our thinking process and creating shortcuts with patterns. At inference time, our model can be configured to output only the solutions (fast mode) or both the reasoning chain and the final solution (slow mode), or automatically decide which mode to engage (auto mode). In all cases, Dualformer outperforms the corresponding baseline models in both performance and computational efficiency: (1) in slow mode, Dualformer optimally solves unseen 30 x 30 maze navigation tasks 97.6% of the time, surpassing the Searchformer (trained on data with complete reasoning traces) baseline performance of 93.3%, while only using 45.5% fewer reasoning steps; (2) in fast mode, Dualformer completes those tasks with an 80% optimal rate, significantly outperforming the Solution-Only model (trained on solution-only data), which has an optimal rate of only 30%. For math problems, our techniques have also achieved improved performance with LLM fine-tuning, showing its generalization beyond task-specific models.

Mastering Chess with a Transformer Model

Mastering Chess with a Transformer Model

Amortized Planning with Large-Scale Transformers: A Case Study on Chess

Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers

Predicting Chess Puzzle Difficulty with Transformers

Chess as a Testbed for Language Model State Tracking

Mastering Chinese Chess AI (Xiangqi) Without Search

Enhancing Chess Reinforcement Learning with Graph Representation

Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models

Multi-Game Decision Transformers

Learning Chess With Language Models and Transformers

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping

Understanding Transformer Reasoning Capabilities via Graph Algorithms

Vision Transformers for Computer Go

Brainformers: Trading Simplicity for Efficiency

Computers and chess masters: The role of AI in transforming elite human performance

Transcendence: Generative Models Can Outperform The Experts That Train Them

Mastering Atari, Go, chess and shogi by planning with a learned model

Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces

SentiMATE: Learning to play Chess through Natural Language Processing