Abstract:How does language inform our downstream thinking? In particular, how do humans make meaning from language--and how can we leverage a theory of linguistic meaning to build machines that think in more human-like ways? In this paper, we propose rational meaning construction, a computational framework for language-informed thinking that combines neural language models with probabilistic models for rational inference. We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought (PLoT)--a general-purpose symbolic substrate for generative world modeling. Our architecture integrates two computational tools that have not previously come together: we model thinking with probabilistic programs, an expressive representation for commonsense reasoning; and we model meaning construction with large language models (LLMs), which support broad-coverage translation from natural language utterances to code expressions in a probabilistic programming language. We illustrate our framework through examples covering four core domains from cognitive science: probabilistic reasoning, logical and relational reasoning, visual and physical reasoning, and social reasoning. In each, we show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings, while Bayesian inference with the generated programs supports coherent and robust commonsense reasoning. We extend our framework to integrate cognitively-motivated symbolic modules (physics simulators, graphics engines, and planning algorithms) to provide a unified commonsense thinking interface from language. Finally, we explore how language can drive the construction of world models themselves. We hope this work will provide a roadmap towards cognitive models and AI systems that synthesize the insights of both modern and classical computational perspectives.

Natural Language Semantics and Computability

A New Semantic Theory of Natural Language

Could Semantics Be Something Else? Philosophical Challenges for Formal Semantics

Meta Semantics: Towards better natural language understanding and reasoning

On the Computation of Meaning, Language Models and Incomprehensible Horrors

Boolean Semantics for Natural Language

A Mathematical Model for Universal Semantics

Logic and computation as combinatorics

Model-theoretic semantics as model-based science

Semantical Investigations on Non-classical Logics with Recovery Operators: Negation

Proof-Theoretic Semantics, a Problem with Negation and Prospects for Modality

Changing a Semantics: Opportunism or Courage?

Models of High-Level Computation

Extending and Implementing the Stable Model Semantics

Changing the logic without changing the subject: the case of computability

Computing machinery and mentality

How To Solve Moral Conundrums with Computability Theory

Model-Theoretic Logic for Mathematical Theory of Semantic Information and Communication

Back Reaction of Strings in Self-Consistent String Cosmology

From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought

Ontology and Formal Semantics - Integration Overdue