Reporting Non-Consensual Intimate Media: An Audit Study of Deepfakes

Li Qiwei,Shihui Zhang,Andrew Timothy Kasper,Joshua Ashkinaze,Asia A. Eaton,Sarita Schoenebeck,Eric Gilbert
2024-09-19
Abstract:Non-consensual intimate media (NCIM) inflicts significant harm. Currently, victim-survivors can use two mechanisms to report NCIM - as a non-consensual nudity violation or as copyright infringement. We conducted an audit study of takedown speed of NCIM reported to X (formerly Twitter) of both mechanisms. We uploaded 50 AI-generated nude images and reported half under X's "non-consensual nudity" reporting mechanism and half under its "copyright infringement" mechanism. The copyright condition resulted in successful image removal within 25 hours for all images (100% removal rate), while non-consensual nudity reports resulted in no image removal for over three weeks (0% removal rate). We stress the need for targeted legislation to regulate NCIM removal online. We also discuss ethical considerations for auditing NCIM on social platforms.
Computers and Society
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to explore and evaluate the effectiveness of two different mechanisms in dealing with Non - Consensual Intimate Media (NCIM). Specifically, the researchers are concerned with: 1. The difference in efficiency between the two reporting mechanisms, namely Digital Millennium Copyright Act (DMCA) for copyright infringement and non - consensual nudity policy, in removing NCIM content. 2. **The impact of the legal framework**: By comparing the effects of these two mechanisms, the study reveals the differences between federal laws (such as DMCA) and platform - internal policies, as well as the impact of these differences on the speed and effectiveness of content removal. #### Research background Non - Consensual Intimate Media (NCIM), including so - called "revenge porn" and sexualized "deepfakes", is a serious form of online sexual abuse. Victims usually experience significant emotional distress and social isolation, and may even lead to suicide or being killed. Therefore, rapid and effective removal of this content is crucial for protecting the privacy and mental health of victims. Currently, victims can report NCIM through two main channels: - Report to social media platforms as a non - consensual nudity violation. - Report to platforms as a copyright infringement act, according to the Digital Millennium Copyright Act (DMCA). #### Research methods To evaluate the effects of these two mechanisms, the researchers conducted the following experiments: - They used generative AI to create 50 realistic nude "deepfake" images and posted these images on the X (formerly Twitter) platform. - Then, they divided these images into two groups of 25 each and reported them in the name of "non - consensual nudity" and "copyright infringement" respectively. - Finally, they systematically collected data on the review process, timeline, and results, and observed continuously for 21 days. #### Research findings The research results show that: - **Copyright infringement mechanism**: All 25 images were successfully removed within 25 hours, with a removal rate of 100%. - **Non - consensual nudity mechanism**: All 25 images were not removed within more than three weeks, with a removal rate of 0%. In addition, all accounts reported for copyright infringement received a temporary ban notice, while accounts reported for non - consensual nudity had no consequences or notices. #### Conclusions and recommendations The study emphasizes the need to develop special legislation to regulate the removal of online NCIM. The current practice of relying on the good will of platforms is not sufficient to protect the privacy rights of victims. In order to effectively address the NCIM problem, mandatory laws similar to copyright laws need to be developed, requiring online platforms to comply and protect individuals' privacy rights. Through this study, the author calls on all sectors of society to pay attention to this issue and promote relevant legislative and policy reforms to better protect the rights and interests of victims. --- I hope this summary can help you understand the main research purposes and findings of this paper. If you have more questions or need further information, please feel free to let me know!