Information Seeking Using AI Assistants

Ebtesam Al Haque,Chris Brown,Thomas D. LaToza,Brittany Johnson

2024-08-08

Abstract:A good portion of a software practitioners' day involves seeking and using information to support task completion. Although the information needs of software practitioners have been studied extensively, the impact of AI-assisted tools on their needs and information-seeking behaviors remains largely unexplored. To addresses this gap, we conducted a mixed-method study to understand AI-assisted information seeking behavior of practitioners and its impact on their perceived productivity and skill development. We found that developers are increasingly using AI tools to support their information seeking, citing increased efficiency as a key benefit. Our findings also amplify caveats that come with effectively using AI tools for information seeking, especially for learning and skill development, such as the importance of foundational developer knowledge that can guide and inform the information provided by AI tools. Our efforts have implications for the effective integration of AI tools into developer workflows as information retrieval and learning aids.

Software Engineering,Human-Computer Interaction

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to explore the impact of AI-assisted tools on information-seeking behavior in the software development process and their effects on developer productivity and skill development. Specifically, the study focuses on the following three research questions:

1. **RQ1**: When, why, and how do developers use AI tools for information seeking?
2. **RQ2**: What is the impact of using AI tools for information seeking on developer productivity?
3. **RQ3**: What is the impact of using AI tools for information seeking on developer skill development?

### Research Background

With the development of AI technology, more and more developers are starting to use AI-assisted tools to support their work. Although previous research has extensively explored developers' information needs, there is still a lack of in-depth research on the role of AI tools in the information-seeking process and their impact on productivity and skill development. Therefore, the authors aim to fill this research gap through a mixed-methods study (including surveys and interviews).

### Main Findings

- **Usage of AI Tools**: Most respondents indicated that they frequently use AI tools (such as ChatGPT and GitHub Copilot) to support information-seeking tasks. The main advantages of these tools are increased efficiency, personalized responses, and being language-agnostic.
- **Information-Seeking Behavior Patterns**: Developers typically use AI tools to understand best practices, discover new libraries or solutions, and explore trade-offs between different libraries and implementations.
- **Verification Techniques**: Although AI tools play a significant role in information seeking, participants also employed various verification methods to ensure the accuracy of the information, such as unit testing, cross-validation, and iterative improvement.
- **Traditional Information-Seeking Methods**: When AI tools fail to solve problems, developers often turn to colleagues for help, as well as resources like StackOverflow and Reddit.
- **Organizational Policies and Practices**: Organizational policies and practices also significantly influence developers' use of AI tools, including restrictions on certain AI tools and the promotion and support of AI technology.

Through this study, the authors hope to provide valuable insights for the effective integration of AI tools into developers' workflows, thereby enhancing productivity and promoting skill development.