Abstract:Current interactive systems with natural language interfaces lack the ability to understand a complex information-seeking request which expresses several implicit constraints at once, and there is no prior information about user preferences e.g.,"find hiking trails around San Francisco which are accessible with toddlers and have beautiful scenery in summer", where output is a list of possible suggestions for users to start their exploration. In such scenarios, user requests can be issued in one shot in the form of a complex and long query, unlike conversational and exploratory search models, where require short utterances or queries are often presented to the system step by step. We have designed and deployed a platform to collect the data from approaching such complex interactive systems. Moreover, despite with the current advancement of generative language models these models suffer from hallucination in providing accurate factual knowledge. All language models are mostly trained in large part on web-scraped data from the past, which usually is not useful for immediate users' needs. In this article, we propose an IA that leverages Large Language Models (LLM) for complex request understanding and makes it interactive using Reinforcement learning that allows intricately refine user requests by making them complete, leading to better retrieval and reduce LLMs hallucination problems for current user needs. To demonstrate the performance of the proposed modeling paradigm, we have adopted various pre-retrieval metrics that capture the extent to which guided interactions with our system yield better retrieval results. Through extensive experimentation, we demonstrated that our method significantly outperforms several robust baselines.

HINTs: Sensemaking on large collections of documents with Hypergraph visualization and INTelligent agents

Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models

InsightSee: Advancing Multi-agent Vision-Language Models for Enhanced Visual Understanding

Vital Insight: Assisting Experts' Sensemaking Process of Multi-modal Personal Tracking Data Using Visualization and LLM

Supporting Sensemaking of Large Language Model Outputs at Scale

AVIS: Autonomous Visual Information Seeking with Large Language Model Agent

VISTopic: A visual analytics system for making sense of large document collections using hierarchical topic modeling

GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understanding

HAIChart: Human and AI Paired Visualization System

Making Large Language Models Interactive: A Pioneer Study on Supporting Complex Information-Seeking Tasks with Implicit Constraints

KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts

How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?

Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization

AGENTiGraph: An Interactive Knowledge Graph Platform for LLM-based Chatbots Utilizing Private Data

Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models

Topic-based Visual Text Summarization and Analysis 1

Towards an understanding and explanation for mixed-initiative artificial scientific text detection

Explainable Search and Discovery of Visual Cultural Heritage Collections with Multimodal Large Language Models

From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Calls

MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models