Astrid Carolus,Martin Koch,Samantha Straka,Marc Erich Latoschik,Carolin Wienrich
Abstract:The goal of the present paper is to develop and validate a questionnaire to assess AI literacy. In particular, the questionnaire should be deeply grounded in the existing literature on AI literacy, should be modular (i.e., including different facets that can be used independently of each other) to be flexibly applicable in professional life depending on the goals and use cases, and should meet psychological requirements and thus includes further psychological competencies in addition to the typical facets of AIL. We derived 60 items to represent different facets of AI Literacy according to Ng and colleagues conceptualisation of AI literacy and additional 12 items to represent psychological competencies such as problem solving, learning, and emotion regulation in regard to AI. For this purpose, data were collected online from 300 German-speaking adults. The items were tested for factorial structure in confirmatory factor analyses. The result is a measurement instrument that measures AI literacy with the facets Use & apply AI, Understand AI, Detect AI, and AI Ethics and the ability to Create AI as a separate construct, and AI Self-efficacy in learning and problem solving and AI Self-management. This study contributes to the research on AI literacy by providing a measurement instrument relying on profound competency models. In addition, higher-order psychological competencies are included that are particularly important in the context of pervasive change through AI systems.
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve
This paper aims to develop and validate a questionnaire for assessing Artificial Intelligence Literacy (AI Literacy). Specifically, the questionnaire needs to meet the following objectives:
1. **Based on Existing Literature**: The questionnaire should be deeply rooted in the existing AI literacy literature.
2. **Modular Design**: The questionnaire should include different modules that can be used independently to suit various goals and usage scenarios in professional life.
3. **Meet Psychometric Requirements**: The questionnaire should meet psychological measurement requirements, including further psychological capabilities, not just typical AI literacy aspects.
4. **Include Psychological Meta-Competencies**: In addition to traditional AI literacy aspects, the questionnaire should also include higher-level psychological meta-competencies such as problem-solving, learning, and emotional regulation.
### Background and Motivation
With the proliferation of Artificial Intelligence (AI) in daily life, the ability to understand and apply AI has become increasingly important. Whether for work or entertainment, interactions with AI systems will become more common. However, the complexity of AI makes it difficult to understand. To act autonomously and participate in future developments in an AI-influenced world, users need to understand what AI is and what it can do. This skill set is commonly referred to as "AI Literacy."
Although the education sector has made efforts to incorporate AI literacy into school curricula and develop teaching methods, many potential users who are already working cannot benefit from school-based AI education. For them, the current level of AI literacy may affect whether they can adapt to new technologies and whether the implementation of AI-dependent workflows is successful. Therefore, fostering AI literacy in work and adult education contexts is also very important.
Currently, methods for measuring AI literacy are limited, and many scales are only applicable to specific contexts (e.g., only used in medical settings) and have not been validated. Additionally, no measurement tools consider psychological meta-competencies, which are particularly important in work and adult education because the introduction of AI systems often accompanies extensive change processes that need to be constructively addressed.
### Research Contribution
This paper systematically develops and validates an AI literacy scale through empirical research, which meets psychometric requirements, has cross-context applicability, is embedded in the current AI literacy literature, and considers psychological meta-competencies. Specifically, the research results provide a measurement tool that measures multiple aspects of AI literacy, including "Using and Applying AI," "Understanding AI," "Detecting AI," "AI Ethics," and "Creating AI" as separate constructs, as well as "AI Self-Efficacy" and "AI Self-Management."
### Conclusion
By developing and validating a comprehensive AI literacy scale, this paper fills a gap in existing research and provides an important measurement tool for AI literacy research. This not only helps in selecting suitable personnel and identifying skill and knowledge gaps but also in evaluating the effectiveness of interventions aimed at improving AI literacy.