Abstract:Instruments such as eye-tracking devices have contributed to understanding how users interact with screen-based search engines. However, user-system interactions in audio-only channels -- as is the case for Spoken Conversational Search (SCS) -- are harder to characterize, given the lack of instruments to effectively and precisely capture interactions. Furthermore, in this era of information overload, cognitive bias can significantly impact how we seek and consume information -- especially in the context of controversial topics or multiple viewpoints. This paper draws upon insights from multiple disciplines (including information seeking, psychology, cognitive science, and wearable sensors) to provoke novel conversations in the community. To this end, we discuss future opportunities and propose a framework including multimodal instruments and methods for experimental designs and settings. We demonstrate preliminary results as an example. We also outline the challenges and offer suggestions for adopting this multimodal approach, including ethical considerations, to assist future researchers and practitioners in exploring cognitive biases in SCS.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address the issue of cognitive bias in Spoken Conversational Search (SCS). Specifically, the paper focuses on the following points: 1. **Understanding and Representing User Behavior and Cognitive Bias in SCS**: - Due to the complete reliance on audio interaction in SCS, there is a lack of effective tools and techniques to capture and analyze user interaction behavior, making it difficult to understand user behavior and cognitive bias in SCS. - Compared to traditional screen-based search engines, there is a significant gap in experimental design and methods for SCS, necessitating the development of new tools and methods. 2. **Detecting and Mitigating the Impact of Cognitive Bias**: - In the era of information overload, cognitive bias has a significant impact on information acquisition and consumption, especially when dealing with controversial topics or multiple viewpoints. - To promote a comprehensive understanding and balanced view of information, methods need to be developed to detect and mitigate these cognitive biases. 3. **Exploring the Application of Multimodal Tools and Methods**: - The paper proposes a framework that combines multimodal tools and methods (such as wearable devices, EEG, etc.) to design experiments and settings to better understand and represent user behavior and cognitive bias in SCS. - Preliminary experimental results demonstrate the potential of using these multimodal tools, such as using EEG as an alternative to eye-tracking in the voice channel. 4. **Discussing Challenges and Ethical Considerations**: - The paper also discusses the challenges and ethical issues faced when adopting multimodal methods, including data privacy, user consent, etc., to ensure the accuracy and representativeness of the research results. In summary, by introducing multimodal tools and methods, this paper aims to fill the research gap in cognitive bias in SCS, providing guidance for future researchers and practitioners to better understand and address cognitive bias issues in SCS.

Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search

Towards Investigating Biases in Spoken Conversational Search

Towards a model for spoken conversational search

Exploring Conversational Agents as an Effective Tool for Measuring Cognitive Biases in Decision-Making

Toward Automated Detection of Biased Social Signals from the Content of Clinical Conversations

Cognitively Biased Users Interacting with Algorithmically Biased Results in Whole-Session Search on Debated Topics

Workshop on Detection and Design for Cognitive Biases in People and Computing Systems

Ethically Collecting Multi-Modal Spontaneous Conversations with People that have Cognitive Impairments

BiasEye: A Bias-Aware Real-time Interactive Material Screening System for Impartial Candidate Assessment

Inclusive Design Insights from a Preliminary Image-Based Conversational Search Systems Evaluation

Fair or Fare? Understanding Automated Transcription Error Bias in Social Media and Videoconferencing Platforms

Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models

Integrating Psychometrics and Computing Perspectives on Bias and Fairness in Affective Computing: A Case Study of Automated Video Interviews

Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Review

Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark

Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks

Evaluating for Evidence of Sociodemographic Bias in Conversational AI for Mental Health Support

Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models

Argumentative Experience: Reducing Confirmation Bias on Controversial Issues through LLM-Generated Multi-Persona Debates

Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants

Characterizing Information Seeking Processes with Multiple Physiological Signals