Abstract:The detection of road signs and other objects in the environment are critical for automated vehicles (AVs). Artificial intelligence (AI) is responsible for ensuring that the sensors capture accurate data and process that information into proper decisions. No systems are 100% perfect, and any errors could compromise safety. Currently, the human driver is still required for AVs in case of potential system failures (SAE, 2016). However, we still lack understanding of users' perception and trust while partnering with AVs especially when the AV is experiencing cyberattacks. This study measured users’ AI knowledge, trust, and examined their understanding of the AI’s recognition of maliciously manipulated stop signs. For this study, we ran 205 participants using an online survey through Qualtrics containing 62 questions. The questions were focused on general AI understanding, exposure to AI, general understanding of AI in the context of AVs, applications of AI in AVs, future AI use for AVs, and trust and opinions on AI. Additionally, we tested their understanding of images of a normal stop sign and a maliciously manipulated stop sign. Participants were asked to rate how much they thought the each of the images was of a stop sign and whether the current AI technology would classify them as a stop sign. The testing of these stop signs used a factorial within-subjects design of agent type (user vs. AI) and image type (original vs. malicious). For the user agent, the statements asked if they believed that the image showed a stop sign or not. For the AI agent, the statements asked if they believed that the AI technology would think that the image showed a stop sign. For the original image, the image was a standard stop sign. For the malicious image, the image was the same stop sign image, but it had been tampered with. The malicious stop image used in the study was generated from the algorithms proposed in (Eykholt et al., 2017), which focused on how intentional perturbations on the images can lead to misclassification by an AI. The malicious stop sign was an intentionally perturbed image meant to prevent AI from the proper classification of the sign. The knowledge ratings showed participants were quite knowledgeable about AI and had a high amount of exposure to AI. These two ratings were correlated, likely because the more time spent with a system leads to a greater understanding of what processes are involved in the system. We also found that participants had an accurate perception of stop signs for both the standard and malicious conditions. Both images were recognizable as a stop sign even though the malicious sign was altered. Participants thought that they were better at judging stop signs than the AI, giving lower ratings for AI than their own ability. This result is similar to prior research that shows people are often likely to rely more on their own abilities than that of an AV or have higher selfconfidence than confidence in automation (Hoff & Bashir, 2015; Kyriakidis et al., 2015; Tenhundfeld et al., 2020). Participants rated AI’s ability to detect the malicious image as a stop sign lower than the original stop sign, which was expected. The perturbation noise added onto the sign was designed to be highly effective for confusing the AI. Participants showed high confidence in the AI (with an average rating of 4.15 out of 5). This result demonstrates a potential mismatch between an AI's capabilities and the beliefs that a potential user might have about that system. Participants accurately rated the AI’s ability to detect the original image significantly higher than the malicious image; however, most participants were fairly certain that the malicious image was identifiable by AI. This result could be a sign of overtrust in the AI (Parasuraman & Riley, 1997) because they are unaware of how the AI system works or its true capabilities.

As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli

Predicting a Malicious Stop Sign: Knowledge, Exposure, Trust in AI

Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes

A Representative Study on Human Detection of Artificially Generated Media Across Countries

Human Perception of Audio Deepfakes

“All Around Me Are Synthetic Faces”: The Mad World of AI-Generated Media

Human Detection of Political Speech Deepfakes across Transcripts, Audio, and Video

Detection of AI-Generated Synthetic Faces

Synthetic Photography Detection: A Visual Guidance for Identifying Synthetic Images Created by AI

AI-synthesized faces are indistinguishable from real faces and more trustworthy

Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds

Identifying and minimising the impact of fake visual media: Current and future directions

Deep learning and synthetic media

Image and Video Generation Using Artificial Intelligence and its Detection

Artifact magnification on deepfake videos increases human detection and subjective confidence

Study of detecting behavioral signatures within DeepFake videos

Deepfake Media Forensics: State of the Art and Challenges Ahead

Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors and Perceptual Insights

Fake It Until You Break It: On the Adversarial Robustness of AI-generated Image Detectors

Analysis of Human Perception in Distinguishing Real and AI-Generated Faces: An Eye-Tracking Based Study

How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception