Abstract:Large Language Models (LLMs) exhibit impressive capabilities across various applications but encounter substantial challenges such as high inference latency, considerable training costs, and the generation of hallucinations. Collaborative decoding between large and small language models (SLMs) presents a promising strategy to mitigate these issues through methods including speculative decoding, contrastive decoding, and emulator or proxy fine-tuning. However, the specifics of such collaborations, particularly from a unified perspective, remain largely unexplored. Inspired by dual-process cognitive theory, we propose a unified framework in this paper, termed Fast and Slow Generating (FS-GEN). Within this framework, LLMs (sometimes along with SLMs) are categorized as System 2 (slow and deliberate), while independent SLMs are designated as System 1 (fast and intuitive). We provide a comprehensive analysis of these collaborative methodologies, elucidating their common properties and shedding light on the differential knowledge capabilities of System 2 versus System 1 through the FS-GEN framework. Our findings indicate that only a small proportion of collaborative interactions (approximately less than 20\% in most instances) are necessary across various methods. These interactions between System 1 and System 2 conform to a scaling law related to the parameter ratios, enabling predictable collaboration. Furthermore, we explore the specific conditions under which collaboration proves most effective, particularly from an uncertainty perspective, offering novel insights that may guide future optimization efforts. Our research underscores that the fundamental distinction between System 1 and System 2 lies in the uncertainty of next token predictions, where interventions by System 2 are crucial to support System 1. Code for Reproduction: <a class="link-external link-https" href="https://github.com/TsinghuaC3I/FS-GEN" rel="external noopener nofollow">this https URL</a>

Predictive Simultaneous Interpretation: Harnessing Large Language Models for Democratizing Real-Time Multilingual Communication

TransLLaMa: LLM-based Simultaneous Translation System

Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models

Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact

Exploring Human-Like Translation Strategy with Large Language Models

Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair

Toward Cultural Interpretability: A Linguistic Anthropological Framework for Describing and Evaluating Large Language Models (LLMs)

Speech Translation with Large Language Models: An Industrial Practice

Large Language Models are Interpretable Learners

Anticipating Future with Large Language Model for Simultaneous Machine Translation

Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model

Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners

Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion

Towards Predictive Communication with Brain-Computer Interfaces integrating Large Language Models

Investigating Decoder-only Large Language Models for Speech-to-text Translation

Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?

Large language models for foreign language acquisition

LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers

Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs