Achieving Narrative Change Through AR: Displacing the Single Story to Create Spatial Justice

Janice Tisha Samuels

2024-05-05

Abstract:The ability of Augmented Reality to overcome the bias of single stories through multidimensionality is explored in the artifacts of a youth gun violence prevention project and its goal of narrative change.

Human-Computer Interaction

What problem does this paper attempt to address?

With or Without Permission: Site-Specific Augmented Reality for Social Justice

Rafael M.L. Silva,Ana María Cárdenas Gasca,Joshua A. Fisher,Erica Principe Cruz,Cinthya Jauregui,Amy Lueck,Fannie Liu,Andrés Monroy-Hernández,Kai Lukoff

2024-05-07

Abstract:Movements for social change are often tied to a particular locale. This makes Augmented Reality (AR), which changes how people perceive their surroundings, a promising technology for social justice. Site-specific AR empowers activists to re-tell the story of a place, with or without permission of its owner. It has been used, for example, to reveal hidden histories, re-imagine problematic monuments, and celebrate minority cultures. However, challenges remain concerning technological ownership and accessibility, scalability, sustainability, and navigating collaborations with marginalized communities and across disciplinary boundaries. This half-day workshop at CHI 2024 seeks to bring together an interdisciplinary group of activists, computer scientists, designers, media scholars, and more to identify opportunities and challenges across domains. To anchor the discussion, participants will each share one example of an artifact used in speculating, designing, and/or delivering site-specific AR experiences. This collection of artifacts will inaugurate an interactive database that can inspire a new wave of activists to leverage AR for social justice.

Human-Computer Interaction
Teaching Linguistic Justice through Augmented Reality

Ashvini Varatharaj,Abigail Welch,Mary Bucholtz,Jin Sook Lee

2024-04-19

Abstract:This position paper presents the AR Language Map, a speculative artifact designed to enhance understanding of linguistic justice among middle and high school students through augmented reality (AR) that allows students to map their linguistic experiences. Through a social justice-oriented academic outreach program aimed at linguistically, economically, and racially minoritized students, academic concepts on language, culture, race, and power are introduced to California middle school and high school students. The curriculum has activities for each lesson plan drawn from students' culturally relevant experiences. By enabling interactive exploration of linguistic justice, this tool aims to foster empathy, challenge linguistic racism, and valorize linguistic diversity. We discuss its conceptualization within the broader context of AR in social justice education. The AR Language Map not only deepens students' understanding of these critical issues but also enables them to become co-creators of their learning experiences.

Human-Computer Interaction,Emerging Technologies
New Philadelphia: using augmented reality to interpret slavery and reconstruction era historical sites

Jonathan Amakawa,Jonathan Westin

DOI: https://doi.org/10.1080/13527258.2017.1378909

2017-11-17

International Journal of Heritage Studies

Abstract:Does a historical site lose its significance or become less worthy of interpretation if there are no surviving buildings? Can technology help present the stories of disadvantaged and disenfranchised groups whose heritage lacks well-preserved architecture or material culture? The emerging technology of augmented reality (AR) offers new ways of designing and shaping the public’s experience when visiting landmarks by enabling an unprecedented means to combine 3D historical visualization with historical landmarks. This especially applies to underrepresented groups whose heritages have not been well served by traditional modes of preservation and interpretation due to a variety of factors. These range from disadvantages relating to material culture to a greater emphasis on intangible heritage which have placed them outside the bounds of what archaeologist Laurajane Smith calls authorised heritage discourse. A project at the New Philadelphia National Historic Landmark, located in Pike County Illinois, seeks to address these issues through AR. The technology, while offering opportunities for historical interpretation, poses challenges in terms of designing AR systems that coordinate content presentation with specific locations as well as developing virtual historical content with varying levels of source materials.

social sciences, interdisciplinary
Creative Experiences for Engaging Communities with Cultural Heritage through Place-based Narratives

Karina Rodriguez Echavarria,Myrsini Samaroudi,Laurie Dibble,Edward Silverton,Sophie Dixon

DOI: https://doi.org/10.1145/3479007

2022-06-30

Journal on Computing and Cultural Heritage

Abstract:This research explores technologically advanced means to enhance audiences’ connection with cultural heritage assets through participatory creative methods that particularly reinforce young people’s sense of identity and well-being during sensitive “transitional” periods of their lives. Hence, the research investigates how communities can meaningfully connect with cultural heritage through creative experiences, while aiming at lowering the entry barriers to increasing audiences’ participation. For this, the research deploys narrative approaches to illuminate different viewpoints and interpretations of cultural heritage within communities. The contribution of the article is twofold, as it includes a novel approach for developing and re-telling communities’ narratives linked to people, objects, sites, and events in the urban landscape. At the same time, it proposes a workflow to digitise and communicate these narratives through Augmented Reality (AR) Maps by proposing methods for digitisation and producing physical printed elements for the experience. These physical elements are then augmented with digital narratives, delivered through Immersive Web technology. This concept is proposed as a means to document and disseminate the narratives in a way that enhances the public understanding and appreciation of objects and sites. The approach was tested with a class of children in a local primary school in Brighton and Hove (UK) to understand its suitability for community engagement, targeting young audiences. The significance of the research is that it demonstrates the potential of the synergy between creative and digital approaches for enabling meaningful engagement with the cultural heritage while improving the well-being of the participants as well as their sense of community and place.

computer science, interdisciplinary applications
The Clarkston AR Gateways Project: Anchoring Refugee Presence and Narratives in a Small Town

Joshua A. Fisher,Fernando Rochaix

2024-04-12

Abstract:This paper outlines the Clarkston AR Gateways Project, a speculative process and artifact entering its second phase, where Augmented Reality (AR) will be used to amplify the diverse narratives of Clarkston, Georgia's refugee community. Focused on anchoring their stories and presence into the town's physical and digital landscapes, the project employs a participatory co-design approach, engaging directly with community members. This placemaking effort aims to uplift refugees by teaching them AR development skills that help them more autonomously express and elevate their voices through public art. The result is hoped to be AR experiences that not only challenge prevailing narratives but also celebrate the tapestry of cultures in the small town. This work is supported through AR's unique affordance for users to situate their experiences as interactive narratives within public spaces. Such site-specific AR interactive stories can encourage interactions within those spaces that shift how they are conceived, perceived, and experienced. This process of refugee-driven AR creation reflexively alters the space and affirms their presence and agency. The project's second phase aims to establish a model adaptable to diverse, refugee-inclusive communities, demonstrating how AR storytelling can be a powerful tool for cultural orientation and celebration.

Human-Computer Interaction,Emerging Technologies
Sociotechnical Considerations for SLAM Anchors in Location-Based AR

Tiffany T. Nguyen,Cinthya Jauregui,Sarah H. Sallee,Mohan R. Chandrasekar,Liam A'Hearn,Dominic J. Woetzel,Pinak Paliwal,Madison Nguyen,Isabella `Amne Gomez,Xinqi Zhang,Lee M. Panich,Danielle M. Heitmuller,Amy Lueck,Kai Lukoff

2024-04-19

Abstract:In this position paper, we explore the power of storytelling and its connection to place through the use of Augmented Reality (AR) technology, particularly within the context of Thámien Ohlone history on the Santa Clara University campus. To do this, we utilized SLAM and 8th Wall to create virtual, location-based experiences that geolocate tribal stories at present-day sites, showcase the living culture of the Thámien Ohlone tribe, and advocate for physical markers that could exist to recognize their story. When doing so, we made sure to select locations that added to the story each stop tells to serve as our anchors. Our research then investigates both the social and technical considerations involved in selecting anchors for AR experiences, using the Thámien Ohlone AR Tour as a case study.

Human-Computer Interaction
A Study on Narrative Design of Augmented Reality Publications

Yanxiang Zhang,Weiwei Zhang

DOI: https://doi.org/10.1007/978-3-319-95270-3_31

2018-01-01

Abstract:This paper combs the pattern of the combination of contents in physical book and augmented reality content in AR publication. Through case studies on narrative patterns which have emerged in different types of augmented reality publications, the authors summarize the respective advantages in the integration between traditional text-narration and AR-narration, and come to some noteworthy points emerged in the narration design. These considerations and discussions can be useful for guiding the design and implementation of augmented reality publication, thus promote the communication effect and social function of AR publication.
SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality

Mengyu Chen,Andres Monroy-Hernandez,Misha Sra

DOI: https://doi.org/10.1109/ismar52148.2021.00045

2021-10-01

Abstract:Short-form digital storytelling has become a popular medium for millions of people to express themselves. Traditionally, this medium uses primarily 2D media such as text (e.g., memes), images (e.g., Instagram), GIFs (e.g., Giphy), and videos (e.g., TikTok, Snapchat). To expand the modalities from 2D to 3D media, we present SceneAR, a smartphone application for creating sequential scene-based micro narratives in augmented reality (AR). What sets SceneAR apart from prior work is its ability to share the scene-based stories as AR content. No longer limited to sharing images or videos, users can now experience narratives in their own physical environments. Additionally, SceneAR affords users the ability to remix AR content, empowering them to collectively build on others’ creations. We asked 18 people to use SceneAR in a three-day study, and based on user interviews, analyses of screen recordings, and the stories they created, we extracted three themes. From these themes and the study overall, we derived six strategies for designers interested in supporting short-form AR narratives.
The AltaVoces project: A digital narrative approach to anti-oppressive social work research with Latino youth

Jenn M Lilly

DOI: https://doi.org/10.1177/14733250211070590

2022-02-23

Qualitative Social Work

Abstract:Although digital narrative methods lend themselves well to participatory, action-oriented inquiry, these relatively new methods also raise questions about potential risks involved in using digital technologies to engage marginalized groups in social work research. This article examines the feasibility, challenges, and opportunities of using digital narrative methods in anti-oppressive social work research (AOSWR) by providing empirical insights from the AltaVoces project—an AOSWR project that used digital narrative methods to engage Latino youth. This case study demonstrates the compatibility and feasibility of digital narrative methods in AOSWR by examining to what extent the AltaVoces project: (1) used methods that center the contexts, voices, and experiences of oppressed peoples, recognizing the social construction of knowledge and the politics inherent in knowledge creation, (2) critically interrogated power arrangements within research relationships and made efforts to form authentic, collaborative relationships and share power with co-researchers, and (3) acknowledged oppressive systems and institutions and reflected a commitment to transforming, dismantling, or abolishing them through the research purpose, process, and products. We found that digital tools offered new possibilities for centering the voices of Latino youth, rebalancing power in research relationships, and connecting knowledge to action through digital products, in alignment with AOSWR, but also introduced new power hierarchies and risks related to producing digital material. The AltaVoces project provides one example of how digital narrative research may be implemented and evaluated using the integrative AOSWR framework, exposing several aspects of digital narrative research that warrant specific attention and presenting practical strategies for doing so.

social work
Location-based AR for Social Justice: Case Studies, Lessons, and Open Challenges

Hope Schroeder,Rob Tokanel,Kyle Qian,Khoi Le

DOI: https://doi.org/10.48550/arXiv.2302.02050

IF: 6.4588

2023-02-04

Human-Computer Interaction

Abstract:Dear Visitor and Charleston Reconstructed were location-based augmented reality (AR) experiences created between 2018 and 2020 dealing with two controversial monument sites in the US. The projects were motivated by the ability of AR to 1) link layers of context to physical sites in ways that are otherwise difficult or impossible and 2) to visualize changes to physical spaces, potentially inspiring changes to the spaces themselves. We discuss the projects' motivations, designs, and deployments. We reflect on how physical changes to the projects' respective sites radically altered their outcomes, and we describe lessons for future work in location-based AR, particularly for projects in contested spaces.
AR for Sexual Violence: Maintaining Ethical Balance While Enhancing Empathy

Chunwei Lin

2024-04-17

Abstract:This study showcases an augmented reality (AR) experience designed to promote gender justice and increase awareness of sexual violence in Taiwan. By leveraging AR, this project overcomes the limitations of offline exhibitions on social issues by motivating the public to participate and enhancing their willingness to delve into the topic. The discussion explores how direct exposure to sexual violence can induce negative emotions and secondary trauma among users. It also suggests strategies for using AR to alleviate such issues, particularly by avoiding simulations of actual incidents.

Human-Computer Interaction
Augmented Voices: An Augmented Reality Experience Highlighting the Social Injustices of Gender-Based Violence in the Muslim South-Asian Diaspora

Hamida Khatri

2024-04-24

Abstract:This paper delves into the distressing prevalence of gender-based violence (GBV) and its deep-seated psychological ramifications, particularly among Muslim South Asian women living in diasporic communities. Despite the gravity of GBV, these women often face formidable barriers in voicing their experiences and accessing support. "Augmented Voices" emerges as a technological beacon, harnessing the potential of augmented reality (AR) to bridge the digital and physical realms through mobile devices, enhancing the visibility of these often-silenced voices. With its technological motivation firmly anchored in the convergence of AR and real-world interactions, "Augmented Voices" offers a digital platform where storytelling acts as a catalyst, bringing to the fore the experiences shared by these women. By superimposing their narratives onto physical locations via Geographic Information System (GIS) Mapping, the application "augments their voices" in the diaspora, providing a conduit for expression and solidarity. This project, currently at its developmental stage, aspires to elevate the stories of GBV victims to a level where their struggles are not just heard but felt, forging a powerful connection between the user and the narrative. It is designed to transcend the limitations of conventional storytelling, creating an "augmented" reality where voices that are often muted by societal constraints can resonate powerfully. The project underscores the urgent imperative to confront GBV, catalyzing societal transformation and fostering robust support networks for those in the margins. It is a pioneering example of how technology can become a formidable ally in the fight for social justice and the empowerment of the oppressed. Additionally, this paper delves into the AR workflow illustrating its relevance and contribution to the broader theme of site-specific AR for social justice.

Human-Computer Interaction,Emerging Technologies
The Digital Docent: XR storytelling for a Living History Museum

Joe Geigel,Kunal Shailesh Shitut,Juilee Decker,Amanda Doherty,Gary Jacobs

DOI: https://doi.org/10.1145/3385956.3422090

2020-11-01

Abstract:In this work, we describe the use of a digital docent: a 3D avatar, presented using virtual and augmented reality, as a means for providing interactive storytelling experiences at a living history museum. To allow flexibility depending on the user’s location and access to technology, the app is designed to provide a common experience supporting a variety of different delivery modalities including AR devices, mobile AR, and VR on the Web.
Understanding Young People's Creative Goals with Augmented Reality

Amna Liaqat,Fannie Liu,Brian Berengard,Jiaxun Cao,Andrés Monroy-Hernández

2024-11-05

Abstract:Young people are major consumers of Augmented Reality (AR) tools like Pokémon GO, but they rarely engage in creating these experiences. Creating with technology gives young people a platform for expressing themselves and making social connections. However, we do not know what young people want to create with AR, as existing AR authoring tools are largely designed for adults. To investigate the requirements for an AR authoring tool, we ran eight design workshops with 17 young people in Argentina and the United States that centered on young people's perspectives and experiences. We identified four ways in which young people want to create with} AR, and contribute the following design implications for designers of AR authoring tools for young people: (1) Blending imagination into AR scenarios to preserve narratives, (2) Making traces of actions visible to foster social presence, (3) Exploring how AR artifacts can serve as invitations to connect with others, and (4) Leveraging information asymmetry to encourage learning about the physical world.

Human-Computer Interaction
Post-Conflict Urban Landscape Storytelling: Two Approaches to Contemporary Virtual Visualisation of Oral Narratives

Ghieth Alkhateeb,Joanna Storie,Mart Külvik

DOI: https://doi.org/10.3390/land13040406

IF: 3.905

2024-03-23

Land

Abstract:Armed conflicts and resulting displacement disrupt people's sense of place, leading to an imbalance in the people–place relationship, exaggerated by rehabilitation efforts that overlook the sense of place among conflict- and displacement-impacted communities. A continuous landscape narrative that extends from pre- to post-conflict times contributes to recreating the essence of lost landscapes and therefore reconnecting their sense of place. Focusing on a Syrian city that hosted internally displaced persons (IDPs) in the aftermath of the Syrian conflict, this study aims to structure a virtual landscape and narrative depiction of conflict-impacted landscapes. This study proposes a storytelling approach for narrative construction and an AI-powered visualisation approach to revive the image of the elusive landscapes. This study utilised qualitative research methods through in-depth, semi-structured interviews for data collection and an online survey for exploring the perception of virtual landscape images generated with AI text-to-image models (DALL.E 2 and Bing Image Creator). This study indicates that narratives, supported by AI visualisation, are reliable for comprehending landscape transformation and changes in the sense of place. The two approaches can serve as rehabilitation initiatives in post-crisis settings to recall images of elusive landscapes to save them from being lost forever.

environmental studies
Digital Narrative, Documents and Interactive Public History

Tristan Revells,Yuzhu Chai

DOI: https://doi.org/10.1007/978-3-030-62516-0_36

2020-01-01

Abstract:To date, the implications and potential of interactive digital narrative have had a limited effect on history as an academic discipline. This project is an attempt to form a dialogue between the practice of historians and the rich scholarship on interactive narrative already undertaken by literary theorists, researchers of interactive systems, scholars of media studies, and practicing creative technologists. Probing the narrative devices common to the production of historical work, this VR project uses Maya, Unity, and a range of visual and aural historical sources from Republican-era China (1912–1949) to offer a digital demonstration of the possibilities of combining interactive digital narrative with long accepted materials and modes of historical production. Specifically, our project focuses on making accessible to a global audience the stories of several scientists and historical factories key to the rise of the renewable energy industry in China during the 1940s. In turn, we hope that we can offer participants in the interactive digital storytelling community a few thoughts on the potential of collaboration with historians via the avenue of “public history”.
Preserving History through Augmented Reality

Annie Yang

2024-04-20

Abstract:Extended reality can weave together the fabric of the past, present, and future. A two-day design hackathon was held to bring the community together through a love for history and a common goal to use technology for good. Through interviewing an influential community elder, Emile Pitre, and referencing his book Revolution to Evolution, my team developed an augmented reality artifact to tell his story and preserve on revolutionary's legacy that impacted the University of Washington's history forever.

Human-Computer Interaction
Augmented Reality in Design Education: Landscape Architecture Studies as AR Experience

Jeremy Kerr,Gillian Lawson

DOI: https://doi.org/10.1111/jade.12227

2019-07-17

Abstract:<p>The rapid and ongoing development of digital technologies continues to create new opportunities for education. Over the last decade this has enabled the establishment of blended learning approaches and online education. More recently, Augmented Reality (AR) has emerged as a unique technology that can transform learning experiences across diverse disciplines. This article outlines the development of an AR prototype, <i>Master of Time</i>, which was created to educate first year students and non‐designers on the foundational principles of landscape architecture. This study examines the learning potential and benefits of AR technology with a focus on creating new practices in digital storytelling across situated experiences. In outlining project outcomes, the authors propose a series of critical design principles, strategies and methodologies for educators to apply when developing AR learning experiences across disciplines. Included within this is a framework for transdisciplinary and co‐design collaboration, which is essential for educators working in the forefront of learning technologies.</p>

education & educational research
Augmented reality as a Thirdspace: Simultaneous experience of the physical and virtual

Rob Eagle

DOI: https://doi.org/10.1007/978-3-031-25906-7_39

2023-03-22

Abstract:With the proliferation of devices that display augmented reality (AR), now is the time for scholars and practitioners to evaluate and engage critically with emerging applications of the medium. AR mediates the way users see their bodies, hear their environment and engage with places. Applied in various forms, including social media, e-commerce, gaming, enterprise and art, the medium facilitates a hybrid experience of physical and digital spaces. This article employs a model of real-and-imagined space from geographer Edward Soja to examine how the user of an AR app navigates the two intertwined spaces of physical and digital, experiencing what Soja calls a 'Third-space'. The article illustrates the potential for headset-based AR to engender such a Thirdspace through the author's practice-led research project, the installation Through the Wardrobe. This installation demonstrates how AR has the potential to shift the way that users view and interact with their world with artistic applications providing an opportunity to question assumptions of social norms, identity and uses of physical space.

Human-Computer Interaction,Multimedia
A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality

Yunshui Jin,Minhua Ma,Yongning Zhu

DOI: https://doi.org/10.1007/s11042-021-11723-0

Abstract:Over the years, the various mediums available for storytelling have progressively expanded, from spoken to written word, then to film, and now to Virtual Reality (VR) and Augmented Reality (AR). In 2016, the cutting-edge Head-Mounted Display (HMD) AR Microsoft HoloLens was released. However, though it has been several years, the quality of the user experience with narration using HMD-based AR technology has been rarely discussed. The present study explored interactive narrative in HMD-based AR regarding different user interfaces and their influence on users' presence, narrative engagement and reflection. Inspired by an existing exhibition at the National Holocaust Centre and Museum in the U.K., a HoloLens narrative application, entitled The AR Journey, was developed by the authors using two different interaction methods, Natural User Interface (NUI) and Graphical User Interface (GUI), which were used to perform an empirical study. As revealed from the results of the between-subject design experiment, NUI exhibited statistically significant advantages in creating presence for users without 3D Role Playing Game (RPG) experience, and GUI was superior in creating presence and increasing narrative engagement for users with 3D RPG experience. As indicated by the results of the interviews, the overall narrative experience in HMD-based AR was acceptable, and the branching narrative design was engaging. However, HoloLens hardware issues, as well as virtuality and reality mismatch, adversely affected user experience. Design guidelines were proposed according to the qualitative results. Supplementary information: The online version contains supplementary material available at 10.1007/s11042-021-11723-0.

Achieving Narrative Change Through AR: Displacing the Single Story to Create Spatial Justice

With or Without Permission: Site-Specific Augmented Reality for Social Justice

Teaching Linguistic Justice through Augmented Reality

New Philadelphia: using augmented reality to interpret slavery and reconstruction era historical sites

Creative Experiences for Engaging Communities with Cultural Heritage through Place-based Narratives

The Clarkston AR Gateways Project: Anchoring Refugee Presence and Narratives in a Small Town

Sociotechnical Considerations for SLAM Anchors in Location-Based AR

A Study on Narrative Design of Augmented Reality Publications

SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality

The AltaVoces project: A digital narrative approach to anti-oppressive social work research with Latino youth

Location-based AR for Social Justice: Case Studies, Lessons, and Open Challenges

AR for Sexual Violence: Maintaining Ethical Balance While Enhancing Empathy

Augmented Voices: An Augmented Reality Experience Highlighting the Social Injustices of Gender-Based Violence in the Muslim South-Asian Diaspora

The Digital Docent: XR storytelling for a Living History Museum

Understanding Young People's Creative Goals with Augmented Reality

Post-Conflict Urban Landscape Storytelling: Two Approaches to Contemporary Virtual Visualisation of Oral Narratives

Digital Narrative, Documents and Interactive Public History

Preserving History through Augmented Reality

Augmented Reality in Design Education: Landscape Architecture Studies as AR Experience

Augmented reality as a Thirdspace: Simultaneous experience of the physical and virtual

A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality