Abstract:GuessWhich is an engaging visual dialogue game that involves interaction between a Questioner Bot (QBot) and an Answer Bot (ABot) in the context of image-guessing. In this game, QBot's objective is to locate a concealed image solely through a series of visually related questions posed to ABot. However, effectively modeling visually related reasoning in QBot's decision-making process poses a significant challenge. Current approaches either lack visual information or rely on a single real image sampled at each round as decoding context, both of which are inadequate for visual reasoning. To address this limitation, we propose a novel approach that focuses on visually related reasoning through the use of a mental model of the undisclosed image. Within this framework, QBot learns to represent mental imagery, enabling robust visual reasoning by tracking the dialogue state. The dialogue state comprises a collection of representations of mental imagery, as well as representations of the entities involved in the conversation. At each round, QBot engages in visually related reasoning using the dialogue state to construct an internal representation, generate relevant questions, and update both the dialogue state and internal representation upon receiving an answer. Our experimental results on the VisDial datasets (v0.5, 0.9, and 1.0) demonstrate the effectiveness of our proposed model, as it achieves new state-of-the-art performance across all metrics and datasets, surpassing previous state-of-the-art models. Codes and datasets from our experiments are freely available at \href{<a class="link-external link-https" href="https://github.com/xubuvd/GuessWhich" rel="external noopener nofollow">this https URL</a>}.

Branching Narratives: Character Decision Points Detection

Multiverse of Greatness: Generating Story Branches with LLMs

TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models

Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?

The Branch Not Taken: Predicting Branching in Online Conversations

"where is this relationship going?": Understanding Relationship Trajectories in Narrative Text

Character is Destiny: Can Large Language Models Simulate Persona-Driven Decisions in Role-Playing?

GENEVA: GENErating and Visualizing branching narratives using LLMs

CHATTER: A Character Attribution Dataset for Narrative Understanding

Modeling Human Mental States with an Entity-based Narrative Graph

"Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding

Telling Stories through Multi-User Dialogue by Modeling Character Relations

Choice-75: A Dataset on Decision Branching in Script Learning

Chitchat as Interference: Adding User Backstories to Task-Oriented Dialogues

A Benchmark for Understanding and Generating Dialogue between Characters in Stories

Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games

M-SENSE: Modeling Narrative Structure in Short Personal Narratives Using Protagonist's Mental Representations

Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives

Multi-Modal Dialogue State Tracking for Playing GuessWhich Game

IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization

Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses