Functorial Language Games for Question Answering

Giovanni de Felice,Elena Di Lavore,Mario Román,Alexis Toumi

DOI: https://doi.org/10.4204/EPTCS.333.21

2021-01-26

Abstract:We present some categorical investigations into Wittgenstein's language-games, with applications to game-theoretic pragmatics and question-answering in natural language processing.

Computation and Language,Computer Science and Game Theory

What problem does this paper attempt to address?

Question–answer Games

Thomas Ågotnes,Johan van Benthem,Hans van Ditmarsch,Stefan Minica

DOI: https://doi.org/10.3166/jancl.21.265-288

2011-01-01

Journal of Applied Non-Classical Logics

Abstract:We propose strategic games wherein the strategies consist of players asking each other questions and answering those questions. We study simplifications of such games wherein two players simultaneously ask each other a question that the opponent is then obliged to answer. The motivation for our research is to model conversation including the dynamics of questions and answers, to provide new links between game theory and dynamic logics of information, and to exploit the dynamic/strategic structure that, we think, lies implicitly inside epistemic models for epistemic languages, and to make that structure an explicit subject of logical study. Our main contributions are: the notion of a two-person question-answer game with information goals, the existence and computation of equilibria for these games, the correspondence with Bayesian games and their equilibria, and a connection between logic and game theory namely the existence of equilibria for positive goal formulae.
Wittgenstein’s Concept of Language Games

Shiva Zaheri Birgani,Mahnaz Soqandi

DOI: https://doi.org/10.33258/biolae.v2i2.280

2020-07-14

Britain International of Linguistics Arts and Education (BIoLAE) Journal

Abstract:Austrian British philosopher Ludwig Wittgenstein is one of the greatest philosophers in the 20th century. He mostly works in analytic philosophical thought in topics as diverse as logic and language, perception and intention, ethic and religion, aesthetic and culture. Philosophers often create their own vocabularies by giving special meanings to ordinary terms and phrases. Wittgenstein coinages the term of “language games” and the ‘private language argument”. His argument on the language is the rules of the use of ordinary language is neither right nor wrong, neither true nor false, the language is merely useful for the particular applications in which they are applied . Language is defined not as a system of representation but as a system of devices for engaging in various sorts of social activity, hence ‘the meaning of the word is its use in the language.
Languages of Play: Towards semantic foundations for game interfaces

Chris Martens,Matthew A. Hammer

DOI: https://doi.org/10.48550/arXiv.1703.05410

2017-03-15

Programming Languages

Abstract:Formal models of games help us account for and predict behavior, leading to more robust and innovative designs. While the games research community has proposed many formalisms for both the "game half" (game models, game description languages) and the "human half" (player modeling) of a game experience, little attention has been paid to the interface between the two, particularly where it concerns the player expressing her intent toward the game. We describe an analytical and computational toolbox based on programming language theory to examine the phenomenon sitting between control schemes and game rules, which we identify as a distinct player intent language for each game.
Playing log(N)-Questions over Sentences

Peter Potash,Kaheer Suleman

DOI: https://doi.org/10.48550/arXiv.1908.04660

2019-08-13

Abstract:We propose a two-agent game wherein a questioner must be able to conjure discerning questions between sentences, incorporate responses from an answerer, and keep track of a hypothesis state. The questioner must be able to understand the information required to make its final guess, while also being able to reason over the game's text environment based on the answerer's responses. We experiment with an end-to-end model where both agents can learn simultaneously to play the game, showing that simultaneously achieving high game accuracy and producing meaningful questions can be a difficult trade-off.

Computation and Language
Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games

Yizhe Zhang,Jiarui Lu,Navdeep Jaitly

2024-02-21

Abstract:Large language models (LLMs) are effective at answering questions that are clearly asked. However, when faced with ambiguous queries they can act unpredictably and produce incorrect outputs. This underscores the need for the development of intelligent agents capable of asking clarification questions to resolve ambiguities effectively. This capability requires complex understanding, state tracking, reasoning and planning over multiple conversational turns. However, directly measuring this can be challenging. In this paper, we offer a surrogate problem which assesses an LLMs's capability to deduce an entity unknown to itself, but revealed to a judge, by asking the judge a series of queries. This \textit{entity-deducing game} can serve as an evaluation framework to probe the conversational reasoning and planning capabilities of language models. We systematically evaluate various LLMs and discover significant differences in their performance on this task. We find that strong LLMs like GPT-4 outperform human players by a large margin. We further employ Behavior Cloning (BC) to examine whether a weaker model is capable of imitating a stronger model and generalizing to data or domains, using only the demonstrations from a stronger model. We finally propose to use Reinforcement Learning to enhance reasoning and planning capacity of Vicuna models through episodes of game playing, which lead to significant performance improvement. We hope that this problem offers insights into how autonomous agents could be trained to behave more intelligently in ambiguous circumstances.

Computation and Language,Artificial Intelligence,Human-Computer Interaction,Machine Learning
Categorical Tools for Natural Language Processing

Giovanni de Felice

DOI: https://doi.org/10.48550/arXiv.2212.06636

2022-12-13

Abstract:This thesis develops the translation between category theory and computational linguistics as a foundation for natural language processing. The three chapters deal with syntax, semantics and pragmatics. First, string diagrams provide a unified model of syntactic structures in formal grammars. Second, functors compute semantics by turning diagrams into logical, tensor, neural or quantum computation. Third, the resulting functorial models can be composed to form games where equilibria are the solutions of language processing tasks. This framework is implemented as part of DisCoPy, the Python library for computing with string diagrams. We describe the correspondence between categorical, linguistic and computational structures, and demonstrate their applications in compositional natural language processing.

Computation and Language,Category Theory
Quantum Games and Synchronicity

Adina Goldberg

2024-09-20

Abstract:In the flavour of categorical quantum mechanics, we extend nonlocal games to allow quantum questions and answers, using quantum sets (special symmetric dagger Frobenius algebras) and the quantum functions of Musto, Reutter, and Verdon [<a class="link-https" data-arxiv-id="1711.07945" href="https://arxiv.org/abs/1711.07945">arXiv:1711.07945</a>]. Equations are presented using a diagrammatic calculus for tensor categories. To this quantum question and answer setting, we extend the standard definitions, including strategies, correlations, and synchronicity, and we use these definitions to extend results about synchronicity. We extend the graph homomorphism (isomorphism) game to quantum graphs, and show it is synchronous (bisynchronous) and that its perfect quantum-commuting (bi)strategies are quantum graph homomorphisms (isomorphisms). Our extended definitions agree with the existing quantum games literature, except in the case of synchronicity.

Quantum Physics,Quantum Algebra
Game Comonads & Generalised Quantifiers

Adam Ó Conghaile,Anuj Dawar

2024-07-22

Abstract:Game comonads, introduced by Abramsky, Dawar and Wang and developed by Abramsky and Shah, give an interesting categorical semantics to some Spoiler-Duplicator games that are common in finite model theory. In particular they expose connections between one-sided and two-sided games, and parameters such as treewidth and treedepth and corresponding notions of decomposition. In the present paper, we expand the realm of game comonads to logics with generalised quantifiers. In particular, we introduce a comonad graded by two parameters $n \leq k$ such that isomorphisms in the resulting Kleisli category are exactly Duplicator winning strategies in Hella's $n$-bijection game with $k$ pebbles. We define a one-sided version of this game which allows us to provide a categorical semantics for a number of logics with generalised quantifiers. We also give a novel notion of tree decomposition that emerges from the construction.

Logic in Computer Science
Language Games and Social Cognition: Revisiting Bruner

Ilaria Grazzani,Jens Brockmeier

DOI: https://doi.org/10.1007/s12124-019-09489-0

IF: 1.156

2019-04-24

Integrative Psychological and Behavioral Science

Abstract:This paper discusses the notion of language games as cultural practices in children's early linguistic and socio-cognitive development. First, we trace the emergence of this concept in Jerome Bruner's experimental and theoretical work at Oxford University in the 1960s, work that was informed by the thinking of Wittgenstein and Austin, amongst others. Second, we provide a systematic historical account of how Bruner has influenced more recent research traditions in developmental psychology, especially in the field of social cognition. Finally, we hone in on one specific approach within this field developed by the <em>Laboratory for Developmental and Educational Studies in Psychology</em> at the University of Milano Bicocca.

psychology, biological
Conventionalization Of Linguistic Categories Under Simple Communicative Constraints

Tao Gong,Andrea Puglisi,Vittorio Loreto,William S-Y. Wang

DOI: https://doi.org/10.1109/CEC.2008.4631017

2008-01-01

Abstract:The language game approach is widely adopted to study conventionalization of linguistic knowledge. Most of contemporary models concentrate on the dynamics of language games in random or predefined social structures, but neglect the role of communicative constraints. This paper adopts one form of language games, the category game, to discuss whether some simple distance-related communicative constraint may affect the conventionalization of linguistic categories. By comparing the simulation results with those based on another form of language games, the naming game, we point out some essential differences between these two games which cause their distinct performances under the same communicative constraint. This study fills the gap between the dynamics of language games in random structures and that in complex networks, and suggests that internal properties of language games may reversely influence communicative constraints and social structures.
Multi-Structural Games and Beyond

Marco Carmosino,Ronald Fagin,Neil Immerman,Phokion Kolaitis,Jonathan Lenchner,Rik Sengupta

2024-10-31

Abstract:Multi-structural (MS) games are combinatorial games that capture the number of quantifiers of first-order sentences. On the face of their definition, MS games differ from Ehrenfeucht-Fraisse (EF) games in two ways: first, MS games are played on two sets of structures, while EF games are played on a pair of structures; second, in MS games, Duplicator can make any number of copies of structures. In the first part of this paper, we perform a finer analysis of MS games and develop a closer comparison of MS games with EF games. In particular, we point out that the use of sets of structures is of the essence and that when MS games are played on pairs of structures, they capture Boolean combinations of first-order sentences with a fixed number of quantifiers. After this, we focus on another important difference between MS games and EF games, namely, the necessity for Spoiler to play on top of a previous move in order to win some MS games. Via an analysis of the types realized during MS games, we delineate the expressive power of the variant of MS games in which Spoiler never plays on top of a previous move. In the second part we focus on simultaneously capturing number of quantifiers and number of variables in first-order logic. We show that natural variants of the MS game do *not* achieve this. We then introduce a new game, the quantifier-variable tree game, and show that it simultaneously captures the number of quantifiers and number of variables. We conclude by generalizing this game to a family of games, the *syntactic games*, that simultaneously capture reasonable syntactic measures and the number of variables.

Logic in Computer Science,Computational Complexity
Ehrenfeucht-Fraïssé Games in Semiring Semantics

Sophie Brinke,Erich Grädel,Lovro Mrkonjić

DOI: https://doi.org/10.48550/arXiv.2308.04910

2023-08-09

Abstract:Ehrenfeucht-Fraïssé games provide a fundamental method for proving elementary equivalence (and equivalence up to a certain quantifier rank) of relational structures. We investigate the soundness and completeness of this method in the more general context of semiring semantics. Motivated originally by provenance analysis of database queries, semiring semantics evaluates logical statements not just by true or false, but by values in some commutative semiring; this can provide much more detailed information, for instance concerning the combinations of atomic facts that imply the truth of a statement, or practical information about evaluation costs, confidence scores, access levels or the number of successful evaluation strategies. There is a wide variety of different semirings that are relevant for provenance analysis, and the applicability of classical logical methods in semiring semantics may strongly depend on the algebraic properties of the underlying semiring. While Ehrenfeucht-Fraïssé games are sound and complete for logical equivalences in classical semantics, and thus on the Boolean semiring, this is in general not the case for other semirings. We provide a detailed analysis of the soundness and completeness of model comparison games on specific semirings, not just for classical Ehrenfeucht-Fraïssé games but also for other variants based on bijections or counting. Finally we propose a new kind of games, called homomorphism games, which are based on the fact that there exist certain rather simple semiring interpretations that are locally very different and can be separated even in a one-move game, but which can be proved to be elementarily equivalent via separating sets of homomorphisms into the Boolean semiring. We prove that these homomorphism games provide a sound and complete method for logical equivalences on finite and infinite lattice semirings.

Logic in Computer Science,Databases,Logic
Question – Answer Games ( extended abstract )

Thomas Ågotnes,Johan van Benthem,Hans van Ditmarsch,Ştefan Minică

2010-01-01

Abstract:I am in the middle of Manhattan and I am lost. I approach a likely stranger on the road and ask him, pointing to the right: “Is this the way to the railway station?” Discounting for a moment the possibility that I will be ignored and that the stranger will continue on his way (of which the—trivial—game move equivalent would be “I prefer not to answer the question”), I will now get one of three possible answers: “Yes,”, “No,”, and “I don’t know.” Each answer is informative. After the third answer I may ask the same question to someone else. Questions and answers can be analyzed in dynamic epistemic logic, and right questions and right answers in game theory. We focus on dynamic epistemic logic and not on game theory, and our main contribution is a novel link between the two. In this introduction we now first address the question, then the answer, and lastly the game wherein such questions and answers figure. The railway example misses that game aspect, but now consider having to decide between asking the conference chair “Are the outcomes of the submissions already known?” or “Is my submission accepted?” What the best question is to ask, also depends on the answers you may expect or fear.
Adversarial Language Games for Advanced Natural Language Intelligence

Yuan Yao,Haoxi Zhong,Zhengyan Zhang,Xu Han,Xiaozhi Wang,Chaojun Xiao,Guoyang Zeng,Zhiyuan Liu,Maosong Sun

DOI: https://doi.org/10.48550/arXiv.1911.01622

2020-12-17

Abstract:We study the problem of adversarial language games, in which multiple agents with conflicting goals compete with each other via natural language interactions. While adversarial language games are ubiquitous in human activities, little attention has been devoted to this field in natural language processing. In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word. The attacker is tasked with inducing the defender to utter the target word invisible to the defender, while the defender is tasked with detecting the target word before being induced by the attacker. In Adversarial Taboo, a successful attacker must hide its intention and subtly induce the defender, while a competitive defender must be cautious with its utterances and infer the intention of the attacker. Such language abilities can facilitate many important downstream NLP tasks. To instantiate the game, we create a game environment and a competition platform. Comprehensive experiments and empirical studies on several baseline attack and defense strategies show promising and interesting results. Based on the analysis on the game and experiments, we discuss multiple promising directions for future research.

Computation and Language
Planning, Inference and Pragmatics in Sequential Language Games

Fereshte Khani,Noah D. Goodman,Percy Liang

DOI: https://doi.org/10.48550/arXiv.1805.11774

2018-05-30

Abstract:We study sequential language games in which two players, each with private information, communicate to achieve a common goal. In such games, a successful player must (i) infer the partner's private information from the partner's messages, (ii) generate messages that are most likely to help with the goal, and (iii) reason pragmatically about the partner's strategy. We propose a model that captures all three characteristics and demonstrate their importance in capturing human behavior on a new goal-oriented dataset we collected using crowdsourcing.

Computation and Language
Quantifiers closed under partial polymorphisms

Anuj Dawar,Lauri Hella

2023-08-08

Abstract:We study Lindstrom quantifiers that satisfy certain closure properties which are motivated by the study of polymorphisms in the context of constraint satisfaction problems (CSP). When the algebra of polymorphisms of a finite structure B satisfies certain equations, this gives rise to a natural closure condition on the class of structures that map homomorphically to B. The collection of quantifiers that satisfy closure conditions arising from a fixed set of equations are rather more general than those arising as CSP. For any such conditions P, we define a pebble game that delimits the distinguishing power of the infinitary logic with all quantifiers that are P-closed. We use the pebble game to show that the problem of deciding whether a system of linear equations is solvable in Z2 is not expressible in the infinitary logic with all quantifiers closed under a near-unanimity condition.

Logic in Computer Science
The Ludii Game Description Language is Universal

Dennis J. N. J. Soemers,Éric Piette,Matthew Stephenson,Cameron Browne

2024-06-13

Abstract:There are several different game description languages (GDLs), each intended to allow wide ranges of arbitrary games (i.e., general games) to be described in a single higher-level language than general-purpose programming languages. Games described in such formats can subsequently be presented as challenges for automated general game playing agents, which are expected to be capable of playing any arbitrary game described in such a language without prior knowledge about the games to be played. The language used by the Ludii general game system was previously shown to be capable of representing equivalent games for any arbitrary, finite, deterministic, fully observable extensive-form game. In this paper, we prove its universality by extending this to include finite non-deterministic and imperfect-information games.

Artificial Intelligence,Computer Science and Game Theory
Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategy

David Schlangen

DOI: https://doi.org/10.48550/arXiv.2304.07007

2023-04-14

Computation and Language

Abstract:How does one measure "ability to understand language"? If it is a person's ability that is being measured, this is a question that almost never poses itself in an unqualified manner: Whatever formal test is applied, it takes place on the background of the person's language use in daily social practice, and what is measured is a specialised variety of language understanding (e.g., of a second language; or of written, technical language). Computer programs do not have this background. What does that mean for the applicability of formal tests of language understanding? I argue that such tests need to be complemented with tests of language use embedded in a practice, to arrive at a more comprehensive evaluation of "artificial language understanding". To do such tests systematically, I propose to use "Dialogue Games" -- constructed activities that provide a situational embedding for language use. I describe a taxonomy of Dialogue Game types, linked to a model of underlying capabilites that are tested, and thereby giving an argument for the \emph{construct validity} of the test. I close with showing how the internal structure of the taxonomy suggests an ordering from more specialised to more general situational language understanding, which potentially can provide some strategic guidance for development in this field.
Autoformalization of Game Descriptions using Large Language Models

Agnieszka Mensfelt,Kostas Stathis,Vince Trencsenyi

2024-09-19

Abstract:Game theory is a powerful framework for reasoning about strategic interactions, with applications in domains ranging from day-to-day life to international politics. However, applying formal reasoning tools in such contexts is challenging, as these scenarios are often expressed in natural language. To address this, we introduce a framework for the autoformalization of game-theoretic scenarios, which translates natural language descriptions into formal logic representations suitable for formal solvers. Our approach utilizes one-shot prompting and a solver that provides feedback on syntactic correctness to allow LLMs to refine the code. We evaluate the framework using GPT-4o and a dataset of natural language problem descriptions, achieving 98% syntactic correctness and 88% semantic correctness. These results show the potential of LLMs to bridge the gap between real-life strategic interactions and formal reasoning.

Artificial Intelligence,Computer Science and Game Theory
Large Language Models and Games: A Survey and Roadmap

Roberto Gallotta,Graham Todd,Marvin Zammit,Sam Earle,Antonios Liapis,Julian Togelius,Georgios N. Yannakakis

DOI: https://doi.org/10.1109/TG.2024.3461510

2024-10-02

Abstract:Recent years have seen an explosive increase in research on large language models (LLMs), and accompanying public engagement on the topic. While starting as a niche area within natural language processing, LLMs have shown remarkable potential across a broad range of applications and domains, including games. This paper surveys the current state of the art across the various applications of LLMs in and for games, and identifies the different roles LLMs can take within a game. Importantly, we discuss underexplored areas and promising directions for future uses of LLMs in games and we reconcile the potential and limitations of LLMs within the games domain. As the first comprehensive survey and roadmap at the intersection of LLMs and games, we are hopeful that this paper will serve as the basis for groundbreaking research and innovation in this exciting new field.

Computation and Language,Artificial Intelligence,Human-Computer Interaction

Functorial Language Games for Question Answering

Question–answer Games

Wittgenstein’s Concept of Language Games

Languages of Play: Towards semantic foundations for game interfaces

Playing log(N)-Questions over Sentences

Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games

Categorical Tools for Natural Language Processing

Quantum Games and Synchronicity

Game Comonads & Generalised Quantifiers

Language Games and Social Cognition: Revisiting Bruner

Conventionalization Of Linguistic Categories Under Simple Communicative Constraints

Multi-Structural Games and Beyond

Ehrenfeucht-Fraïssé Games in Semiring Semantics

Question – Answer Games ( extended abstract )

Adversarial Language Games for Advanced Natural Language Intelligence

Planning, Inference and Pragmatics in Sequential Language Games

Quantifiers closed under partial polymorphisms

The Ludii Game Description Language is Universal

Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategy

Autoformalization of Game Descriptions using Large Language Models

Large Language Models and Games: A Survey and Roadmap