To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming

Ryan Yen,Nicole Sultanum,Jian Zhao

2024-02-02

Abstract:The convergence of generative AI and web search is reshaping problem-solving for programmers. However, the lack of understanding regarding their interplay in the information-seeking process often leads programmers to perceive them as alternatives rather than complementary tools. To analyze this interaction and explore their synergy, we conducted an interview study with eight experienced programmers. Drawing from the results and literature, we have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a comprehensive process model that captures programmers' interaction patterns. This model encompasses decision-making stages, the information-foraging loop, and cognitive activities during system interaction, offering a holistic framework to comprehend and optimize the use of these convergent tools in programming.

Human-Computer Interaction

What problem does this paper attempt to address?

The paper attempts to address the issue of understanding the synergy between generative AI and web search in programming and exploring how they can jointly optimize the information-seeking process for programmers. Specifically, the paper focuses on the following aspects:

1. **Tool Selection and Integration**: Currently, many programmers tend to view generative AI and web search as alternative tools rather than complementary ones when handling programming tasks. The paper aims to reveal the synergistic effects between these two tools through research, helping programmers better understand when to use which tool and how to effectively combine them.

2. **Decision-Making Process**: Through interview studies, the paper identifies three main challenges programmers face in the information-seeking process and proposes three decision-making stages, each with its specific influencing factors. These challenges include:
- **Lack of guidance in determining tool selection and integration** (C1): Programmers need help deciding which tool to use in different situations.
- **Lack of a framework for extracting information** (C2): Programmers need better methods to evaluate and extract information obtained from results.
- **Difficulty in converting results between search and generative AI** (C3): Programmers find it challenging to convert results from one format to another when switching between the two tools.

3. **Comprehensive Process Model**: Based on the research findings, the paper proposes a comprehensive process model that covers various stages of interaction between programmers and generative AI and web search. This model includes decision-making stages, information-seeking loops, and cognitive activities during system interaction, providing a comprehensive framework for future design to help programmers more effectively utilize these two tools.

Through this research, the paper aims to provide guidance for future tool design, enabling generative AI and web search to work better together, thereby improving programmers' work efficiency and problem-solving capabilities.