Visual Agents as Fast and Slow Thinkers

Guangyan Sun,Mingyu Jin,Zhenting Wang,Cheng-Long Wang,Siqi Ma,Qifan Wang,Ying Nian Wu,Yongfeng Zhang,Dongfang Liu

2024-08-17

Abstract:Achieving human-level intelligence requires refining cognitive distinctions between System 1 and System 2 thinking. While contemporary AI, driven by large language models, demonstrates human-like traits, it falls short of genuine cognition. Transitioning from structured benchmarks to real-world scenarios presents challenges for visual agents, often leading to inaccurate and overly confident responses. To address the challenge, we introduce FaST, which incorporates the Fast and Slow Thinking mechanism into visual agents. FaST employs a switch adapter to dynamically select between System 1/2 modes, tailoring the problem-solving approach to different task complexity. It tackles uncertain and unseen objects by adjusting model confidence and integrating new contextual data. With this novel design, we advocate a flexible system, hierarchical reasoning capabilities, and a transparent decision-making pipeline, all of which contribute to its ability to emulate human-like cognitive processes in visual intelligence. Empirical results demonstrate that FaST outperforms various well-known baselines, achieving 80.8% accuracy over VQA^{v2} for visual question answering and 48.7% GIoU score over ReasonSeg for reasoning segmentation, demonstrate FaST's superior performance. Extensive testing validates the efficacy and robustness of FaST's core components, showcasing its potential to advance the development of cognitive visual agents in AI systems.

Machine Learning

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper primarily addresses the issues faced by visual agents when transitioning from structured benchmarks to complex real-world scenarios. Specifically:

1. **Cognitive Distinction**:
- Current AI systems, especially those based on large-scale language models (LLMs), exhibit human-like characteristics but lack true cognitive abilities.
- The paper proposes the need to clearly distinguish between fast thinking (System 1) and slow thinking (System 2) cognitive modes in AI systems.

2. **Inaccurate and Overconfident Responses**:
- Existing multimodal large language models (MLLMs) often provide overly confident but inaccurate answers when dealing with complex problems.
- This issue stems from the lack of explicit modeling of fast and slow cognitive processes.

3. **Introduction of the FAST Mechanism**:
- To address these issues, the paper introduces the FAST mechanism (Fast and Slow Thinking), which dynamically selects either the fast or slow thinking mode to solve problems of varying complexity.
- The FAST mechanism includes a switch adapter that dynamically selects the appropriate thinking mode based on task difficulty and handles uncertainty and unseen objects by adjusting model confidence and integrating new contextual data.

Through these improvements, FAST achieves significantly better performance than existing methods on multiple benchmarks, particularly excelling in tasks such as Visual Question Answering (VQA) and Reasoning Segmentation. These enhancements help improve the cognitive abilities and decision-making transparency of visual agents.