Abstract:The ability to learn abstract concepts is a powerful component of human cognition. It has been argued that variable binding is the key element enabling this ability, but the computational aspects of variable binding remain poorly understood. Here, we address this shortcoming by formalizing the Hierarchical Language of Thought (HLOT) model of rule learning. Given a set of data items, the model uses Bayesian inference to infer a probability distribution over stochastic programs that implement variable binding. Because the model makes use of symbolic variables as well as Bayesian inference and programs with stochastic primitives, it combines many of the advantages of both symbolic and statistical approaches to cognitive modeling. To evaluate the model, we conducted an experiment in which human subjects viewed training items and then judged which test items belong to the same concept as the training items. We found that the HLOT model provides a close match to human generalization patterns, significantly outperforming two variants of the Generalized Context Model, one variant based on string similarity and the other based on visual similarity using features from a deep convolutional neural network. Additional results suggest that variable binding happens automatically, implying that binding operations do not add complexity to peoples' hypothesized rules. Overall, this work demonstrates that a cognitive model combining symbolic variables with Bayesian inference and stochastic program primitives provides a new perspective for understanding people's patterns of generalization.

Discovering Variable Binding Circuitry with Desiderata

Learning abstract visual concepts via probabilistic program induction in a Language of Thought

Uncovering Intermediate Variables in Transformers using Circuit Probing

Hypothesizing Missing Causal Variables with LLMs

Representational Analysis of Binding in Large Language Models

Variable binding and substitution for (nameless) dummies

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models

From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control

Automatically Identifying Local and Global Circuits with Linear Computation Graphs

Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors

Revisiting Variable Ordering for Real Quantifier Elimination using Machine Learning

Discovering and Orienting the Edges Connected to a Target Variable in a DAG Via a Sequential Local Learning Approach.

Leveraging Language to Learn Program Abstractions and Search Heuristics

TaskLAMA: Probing the Complex Task Understanding of Language Models

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Causal interventions expose implicit situation models for commonsense language understanding

Uncovering Latent Chain of Thought Vectors in Language Models

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT

Representational Analysis of Binding in Language Models