Assessing AI Impact Assessments: A Classroom Study

Nari Johnson,Hoda Heidari

2023-11-19

Abstract:Artificial Intelligence Impact Assessments ("AIIAs"), a family of tools that provide structured processes to imagine the possible impacts of a proposed AI system, have become an increasingly popular proposal to govern AI systems. Recent efforts from government or private-sector organizations have proposed many diverse instantiations of AIIAs, which take a variety of forms ranging from open-ended questionnaires to graded score-cards. However, to date that has been limited evaluation of existing AIIA instruments. We conduct a classroom study (N = 38) at a large research-intensive university (R1) in an elective course focused on the societal and ethical implications of AI. We assign students to different organizational roles (for example, an ML scientist or product manager) and ask participant teams to complete one of three existing AI impact assessments for one of two imagined generative AI systems. In our thematic analysis of participants' responses to pre- and post-activity questionnaires, we find preliminary evidence that impact assessments can influence participants' perceptions of the potential risks of generative AI systems, and the level of responsibility held by AI experts in addressing potential harm. We also discover a consistent set of limitations shared by several existing AIIA instruments, which we group into concerns about their format and content, as well as the feasibility and effectiveness of the activity in foreseeing and mitigating potential harms. Drawing on the findings of this study, we provide recommendations for future work on developing and validating AIIAs.

Computers and Society,Artificial Intelligence,Human-Computer Interaction

What problem does this paper attempt to address?

The paper aims to explore the effectiveness and usability of Artificial Intelligence Impact Assessments (AIIAs) and evaluate existing AIIAs templates through classroom research. Specifically, the objectives of the paper include the following aspects:

1. **Explore the usability and effectiveness of existing AIIAs templates**: By designing a role-playing activity, 38 students will play different organizational roles (such as machine learning scientists, product managers, or user representatives) and complete one of three different AIIAs templates to evaluate two hypothetical generative AI systems.

2. **Analyze the impact of AIIAs on students' thinking**: Through pre- and post-questionnaire comparisons, the study will investigate whether students' awareness of potential risks, attitudes towards generative AI, and sense of responsibility as machine learning experts change after completing the AIIAs.

3. **Identify the limitations of existing AIIAs tools**: By analyzing student feedback, the study will uncover the shortcomings of current AIIAs tools in terms of format, content, feasibility, and effectiveness, and propose improvement suggestions.

Through the above research, the paper hopes to provide guidance for the future development and validation of more effective AIIAs.