VIF: Virtual Interactive Fiction (with a twist)

Jérémy Frey

DOI: https://doi.org/10.48550/arXiv.1606.02427

2016-06-08

Abstract:Nowadays computer science can create digital worlds that deeply immerse users; it can also process in real time brain activity to infer their inner states. What marvels can we achieve with such technologies? Go back to displaying text. And unfold a story that follows and molds users as never before.

Human-Computer Interaction

What problem does this paper attempt to address?

Interactive Fiction Games: A Colossal Adventure

Matthew Hausknecht,Prithviraj Ammanabrolu,Marc-Alexandre Côté,Xingdi Yuan

DOI: https://doi.org/10.1609/aaai.v34i05.6297

2020-04-03

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:A hallmark of human intelligence is the ability to understand and communicate with language. Interactive Fiction games are fully text-based simulation environments where a player issues text commands to effect change in the environment and progress through the story. We argue that IF games are an excellent testbed for studying language-based autonomous agents. In particular, IF games combine challenges of combinatorial action spaces, language understanding, and commonsense reasoning. To facilitate rapid development of language-based agents, we introduce Jericho, a learning environment for man-made IF games and conduct a comprehensive study of text-agents across a rich set of games, highlighting directions in which agents can improve.
Immersed in my Ideas: Using Virtual Reality and Multimodal Interactions to Visualize Users' Ideas and Thoughts

Yunhao Xing,Jerrick Ban,Timothy D. Hubbard,Michael Villano,Diego Gomez-Zara

2024-09-23

Abstract:This paper introduces VIVRA (Voice Interactive Virtual Reality Annotation), a VR application combining multimodal interaction with large language models (LLMs) to transform users' ideas into interactive 3D visualizations. VIVRA converts verbalized thoughts into "idea balloons" that summarize and expand on detected topics by an LLM. VIVRA allows users to verbalize their thoughts in real time or record their ideas to display the topics later. We evaluated the effectiveness of VIVRA in an exploratory study with 29 participants and a user study with 10 participants. Our results show that VIVRA enhanced users' ability to reflect on and develop ideas, achieving high levels of satisfaction, usability, and engagement. Participants valued VIVRA as a reflective tool for exploring personal thoughts and ideas. We discuss the potential advantages and uses of this application, highlighting the potential of combining immersive technologies with LLMs to create powerful ideation and reflection tools.

Human-Computer Interaction
Physiologically Driven Storytelling: Concept and Software Tool

Jérémy Frey,Gilad Ostrin,May Grabli,Jessica R. Cauchard

DOI: https://doi.org/10.1145/3313831.3376643

2020-04-21

Abstract:We put forth Physiologically Driven Storytelling, a new approach to interactive storytelling where narratives adaptively unfold based on the reader's physiological state. We first describe a taxonomy framing how physiological signals can be used to drive interactive systems both as input and output. We then propose applications to interactive storytelling and describe the implementation of a software tool to create Physiological Interactive Fiction (PIF). The results of an online study (N=140) provided guidelines towards augmenting the reading experience. PIF was then evaluated in a lab study (N=14) to determine how physiological signals can be used to infer a reader's state. Our results show that breathing, electrodermal activity, and eye tracking can help differentiate positive from negative tones, and monotonous from exciting events. This work demonstrates how PIF can support storytelling in creating engaging content and experience tailored to the reader. Moreover, it opens the space to future physiologically driven systems within broader application areas.
V-IRL: Grounding Virtual Intelligence in Real Life

Jihan Yang,Runyu Ding,Ellis Brown,Xiaojuan Qi,Saining Xie

2024-07-18

Abstract:There is a sensory gulf between the Earth that humans inhabit and the digital realms in which modern AI agents are created. To develop AI agents that can sense, think, and act as flexibly as humans in real-world settings, it is imperative to bridge the realism gap between the digital and physical worlds. How can we embody agents in an environment as rich and diverse as the one we inhabit, without the constraints imposed by real hardware and control? Towards this end, we introduce V-IRL: a platform that enables agents to scalably interact with the real world in a virtual yet realistic environment. Our platform serves as a playground for developing agents that can accomplish various practical tasks and as a vast testbed for measuring progress in capabilities spanning perception, decision-making, and interaction with real-world data across the entire globe.

Artificial Intelligence,Computer Vision and Pattern Recognition
Virtual Human: Storytelling and Computer Graphics for a Virtual Human Platform

Stefan Göbel,Oliver Schneider,Ido Iurgel,Axel Feix,Christian Knöpfle,Alexander Rettig

DOI: https://doi.org/10.1007/978-3-540-27797-2_11

2004-01-01

Abstract:This paper describes the usage of Computer Graphics and Interactive, Digital Storytelling Concepts within the Virtual Human project and its technical platform. Based on a brief overview of the Virtual Human approach, global aims and first R&D results, the Virtual Human platform is introduced. Hereby, methods and concepts for the authoring environment, the narration engine and the avalon player as rendering platform are provided as well as an overview of the Direction and Player Markup Language used for interfaces purposes between these components. Finally, our current Virtual Human demonstrator recently presented at CeBIT 2004 in Hannover using these methods and concepts is described and further R&D activities are pointed out within a brief summary and outlook.
Virtual reality as a tool for promoting reading via enhanced narrative absorption and empathy

Federico Pianzola,Katalin Bálint,Jessica Weller

DOI: https://doi.org/10.1075/ssol.19013.pia

2019-12-31

Scientific Study of Literature

Abstract:Abstract Reading fiction is beneficial for various social skills, although reading has become less and less popular with younger generations. This study investigated whether reading a chapter of a fictional story in virtual reality (VR) can make the reading experience more appealing and increase intention to read the story further. A between-subject experiment ( N = 83) was conducted to compare the effect of a printed book and a VR reading environment on narrative absorption, empathy with fictional characters, and intention to read. The results show that VR enhances intention to read, via a serial mediation of transportation into the story world and affective empathy. These findings indicate that VR can be effectively exploited for promoting reading.
iFiction: Mobile technology, new media, Mixed Reality and literary creativity in English teaching

A. Thomas,W. Chinthammit

DOI: https://doi.org/10.1109/ISMAR-AMH.2012.6483987

2012-11-01

Abstract:iFiction is a location-based mobile application using Mixed Reality and multimedia technologies designed to enhance and transform children's interactive, participatory and innovative experiences with English literature. With this app, children author their own interactive reversionings of literary texts using a range of traditional multimedia technologies (such as text, audio, photo, video) as well as Mixed Reality (MR) technologies and then manipulate these media into a MR layering of the story that can be placed within a particular context of space and time, blending virtual and actual reality. iFiction uses theories of multimodal authoring and principles drawn from alternate reality gaming to create a unique context for children to learn to be sophisticated content creators, and to engage in critical and reflective cultural practice. We will report preliminary findings from our pilot project with school children as an indicative proof of concept of iFiction.

Computer Science,Education
EntangleVR++: Evaluating the Potential of using Entanglement in an Interactive VR Scene Creation System

Mengyu Chen,Marko Peljhan,Misha Sra

DOI: https://doi.org/10.3389/frvir.2023.1252551

2024-06-23

Abstract:Interactive digital stories provide a sense of flexibility and freedom to players by allowing them to make choices at key junctions. These choices advance the narrative and determine, to some degree, how the story evolves for that player. As shown in prior work, the ability to control or participate in the construction of the narrative can give the player a high level of agency that results in a stronger sense of immersion in the narrative experience. To support the design of this type of interactive storytelling, our system, EntangleVR++, borrows the idea of entanglement from quantum computing. Our use of entanglement allows creators and storytellers control over which sequences of story events take place in correlation with each other, initiated by the choices a player makes. In this work, we evaluated how well our idea of entanglement enables creators to easily and quickly design interactive VR narratives. We asked 16 participants to use our system and based on user interviews, analyses of screen recordings, and questionnaire feedback, we extracted four themes. From these themes and the study overall, we derived four authoring strategies for tool designers interested in the design of future visual interface for interactively creating virtual scenes that include relational objects and multiple outcomes driven by player interactions.

Human-Computer Interaction
Unrolling Virtual Worlds for Immersive Experiences

Alexey Tikhonov,Anton Repushko

DOI: https://doi.org/10.48550/arXiv.2311.17924

2023-11-14

Graphics

Abstract:This research pioneers a method for generating immersive worlds, drawing inspiration from elements of vintage adventure games like Myst and employing modern text-to-image models. We explore the intricate conversion of 2D panoramas into 3D scenes using equirectangular projections, addressing the distortions in perception that occur as observers navigate within the encompassing sphere. Our approach employs a technique similar to "inpainting" to rectify distorted projections, enabling the smooth construction of locally coherent worlds. This provides extensive insight into the interrelation of technology, perception, and experiential reality within human-computer interaction.
VRFromX: from Scanned Reality to Interactive Virtual Experience with Human-in-the-Loop.

Ananya Ipsita,Hao Li,Runlin Duan,Yuanzhi Cao,Subramanian Chidambaram,Min Liu,Karthik Ramani

DOI: https://doi.org/10.1145/3411763.3451747

2021-01-01

Abstract:There is an increasing trend of Virtual-Reality (VR) applications found in education, entertainment, and industry. Many of them utilize real world tools, environments, and interactions as bases for creation. However, creating such applications is tedious, fragmented, and involves expertise in authoring VR using programming and 3D-modelling softwares. This hinders VR adoption by decoupling subject matter experts from the actual process of authoring while increasing cost and time. We present VRFromX, an in-situ Do-It-Yourself (DIY) platform for content creation in VR that allows users to create interactive virtual experiences. Using our system, users can select region(s) of interest (ROI) in scanned point cloud or sketch in mid-air using a brush tool to retrieve virtual models and then attach behavioral properties to them. We ran an exploratory study to evaluate usability of VRFromX and the results demonstrate feasibility of the framework as an authoring tool. Finally, we implemented three possible use-cases to showcase potential applications.
Virtual Psychedelia

Jacob Yenney,Weichen Liu,Ying C. Wu

2024-05-02

Abstract:We present an approach to designing 3D Iterated Function Systems (IFS) within the Unity Editor and rendered to VR in real-time. Objects are modeled as a hierarchical tree of primitive shapes and operators, editable using a graphical user interface allowing artists to develop psychedelic scenes with little to no coding knowledge, and is easily extensible for more advanced users to add their own primitive shapes and operators.

Graphics
Design insights into embedding virtual reality content into life skills training for people with intellectual disability

Ross Brown,Laurianne Sitbon,Lauren Fell,Stewart Koplick,Chris Beaumont,Margot Brereton

DOI: https://doi.org/10.1145/3010915.3010956

2016-01-01

Abstract:This paper presents some lessons learnt from the co-creation, within a support organisation, of a virtual world dedicated to practicing life skills for people with intellectual disability. The progression of the project throughout successive stages has indicated what people would want to see and practice in a such a virtual world, which would not have been found after a single development iteration. These desires include the representation of public spaces, social interactions and disruptions. Based on initial trials, we propose some new design insights for further implementation of such virtual worlds in training sessions with the use of immersive headsets. While immersive headsets have been demonstrated to enhance the learning experience, they also visually isolate the person in training from the trainer. To overcome this isolation, a new design concept is presented which allows the teacher to communicate, using gestures, with the immersed student.
Theoretical limits of Virtual Reality

Francesco Sisini,Valentina Sisini,Laura Sisini

DOI: https://doi.org/10.48550/arXiv.2302.10190

IF: 6.4588

2023-02-18

Human-Computer Interaction

Abstract:In recent years there has been a strong development of the concept of virtual reality (VR) which from the first video games developed in the 60s has reached the current immersive systems. VR and the consequent deception of perception pose an interesting question: is it possible to deceive the mind to the point of not being able to recognize whether the perceived reality is real or simulated? In addition to this question, another question arises spontaneously: is it possible to simulate a non-reality in which the physical laws do not apply? The answer to the first question is that it would theoretically be possible to deceive the mind to the point of not being able to recognize whether the perceived reality is real or simulated, furthermore it is also possible to simulate a non-real, i.e. magical, world. However, this possibility is based on hiding degrees of freedom from the observer, therefore it requires the complicity of the observer to be realised. It can be said that a VR system can simulate both a real and non-real experience. For an observer, a virtual reality experience is still an experience of reality. This experience can be exchanged for a different experience, that is, for a different reality that can be real or not real.
Social Conjuring: Multi-User Runtime Collaboration with AI in Building Virtual 3D Worlds

Amina Kobenova,Cyan DeVeaux,Samyak Parajuli,Andrzej Banburski-Fahey,Judith Amores Fernandez,Jaron Lanier

2024-10-03

Abstract:Generative artificial intelligence has shown promise in prompting virtual worlds into existence, yet little attention has been given to understanding how this process unfolds as social interaction. We present Social Conjurer, a framework for AI-augmented dynamic 3D scene co-creation, where multiple users collaboratively build and modify virtual worlds in real-time. Through an expanded set of interactions, including social and tool-based engagements as well as spatial reasoning, our framework facilitates the creation of rich, diverse virtual environments. Findings from a preliminary user study (N=12) provide insight into the user experience of this approach, how social contexts shape the prompting of spatial environments, and perspective on social applications of prompt-based 3D co-creation. In addition to highlighting the potential of AI-supported multi-user world creation and offering new pathways for AI-augmented creative processes in VR, this article presents a set of implications for designing human-centered interfaces that incorporate AI models into 3D content generation.

Human-Computer Interaction,Artificial Intelligence,Computation and Language,Emerging Technologies
What You Write Is What You Get: A Novel Mixed Reality Interface

ZhiYing Zhou,Syed Omer Gilani,Stefan Winkler

2007-01-01

Abstract:Kids develop their understanding of the world by interacting with different objects. This cognitive development process can be further enriched through the use of interactive multimedia technology. This paper presents a novel application that combines the flexibility of the virtual world with a handwriting recognition engine to create a tangible mixed reality learning environment.
The Character as Subjective Interface

Jonathan Lessard,Dominic Arsenault

DOI: https://doi.org/10.1007/978-3-319-48279-8_28

2016-01-01

Abstract:This paper re-frames virtual interactive characters as “subjective interfaces” with the purpose of highlighting original affordances for interactive storytelling through conversation. This notion is theoretically unpacked in the perspectives of narratology, interaction design and game design. Existing and imagined scenarios are presented in which subjective interfaces are elevated as core interaction mechanics. Finally, technical challenges posed by this approach are reviewed alongside relevant existing research leads.
Tangible Web: An Interactive Immersion Virtual RealityCreativity System that Travels Across Reality

Simin Yang,Ze Gao,Reza Hadi Mogavi,Pan Hui,Tristan Braud

DOI: https://doi.org/10.1145/3543507.3587432

2023-04-05

Abstract:With the advancement of virtual reality (VR) technology, virtual displays have become integral to how museums, galleries, and other tourist destinations present their collections to the public. However, the current lack of immersion in virtual reality displays limits the user's ability to experience and appreciate its aesthetics. This paper presents a case study of a creative approach taken by a tourist attraction venue in developing a physical network system that allows visitors to enhance VR's aesthetic aspects based on environmental parameters gathered by external sensors. Our system was collaboratively developed through interviews and sessions with twelve stakeholder groups interested in art and exhibitions. This paper demonstrates how our technological advancements in interaction, immersion, and visual attractiveness surpass those of earlier virtual display generations. Through multimodal interaction, we aim to encourage innovation on the Web and create more visually appealing and engaging virtual displays. It is hoped that the greater online art community will gain fresh insight into how people interact with virtual worlds as a result of this work.

Human-Computer Interaction,Multimedia
Virtual Laboratories and Virtual Worlds

Piet Hut

DOI: https://doi.org/10.1017/S1743921308016153

2007-12-11

Abstract:Since we cannot put stars in a laboratory, astrophysicists had to wait till the invention of computers before becoming laboratory scientists. For half a century now, we have been conducting experiments in our virtual laboratories. However, we ourselves have remained behind the keyboard, with the screen of the monitor separating us from the world we are simulating. Recently, 3D on-line technology, developed first for games but now deployed in virtual worlds like Second Life, is beginning to make it possible for astrophysicists to enter their virtual labs themselves, in virtual form as avatars. This has several advantages, from new possibilities to explore the results of the simulations to a shared presence in a virtual lab with remote collaborators on different continents. I will report my experiences with the use of Qwaq Forums, a virtual world developed by a new company (see <a class="link-external link-http" href="http://www.qwaq.com" rel="external noopener nofollow">this http URL</a>)

Astrophysics,Human-Computer Interaction,Computational Physics
ConnectVR: A Trigger-Action Interface for Creating Agent-based Interactive VR Stories

Mengyu Chen,Marko Peljhan,Misha Sra

DOI: https://doi.org/10.1109/VR58804.2024.00051

2024-06-23

Abstract:The demand for interactive narratives is growing with increasing popularity of VR and video gaming. This presents an opportunity to create interactive storytelling experiences that allow players to engage with a narrative from a first person perspective, both, immersively in VR and in 3D on a computer. However, for artists and storytellers without programming experience, authoring such experiences is a particularly complex task as it involves coding a series of story events (character animation, movements, time control, dialogues, etc.) to be connected and triggered by a variety of player behaviors. In this work, we present ConnectVR, a trigger-action interface to enable non-technical creators design agent-based narrative experiences. Our no-code authoring method specifically focuses on the design of narratives driven by a series of cause-effect relationships triggered by the player's actions. We asked 15 participants to use ConnectVR in a preliminary workshop study as well as two artists to extensively use our system to create VR narrative projects in a three-week in-depth study. Our findings shed light on the creative opportunities facilitated by ConnectVR's trigger-action approach, particularly its capability to establish chained behavioral effects between virtual characters and objects. The results of both studies underscore the positive feedback from participants regarding our system's capacity to not only support creativity but also to simplify the creation of interactive narrative experiences. Results indicate compatibility with non-technical narrative creator's workflows, showcasing its potential to enhance the overall creative process in the realm of VR narrative design.

Human-Computer Interaction
Reality check. The issue of social plausibility in virtual reality therapy with addiction patients

Céline Borelle,Elsa Forner

DOI: https://doi.org/10.1016/j.socscimed.2024.116653

IF: 5.379

2024-02-10

Social Science & Medicine

Abstract:In France, virtual reality has been experimented since 2016 to address addictive behaviors. The existing literature insists on the "immersive" dimension of the technology as a significant factor in the efficacy of exposure. An ethnographic approach, however, suggests that the realism criterion should be reassessed. Based on a fieldwork conducted in the addictology department of a French hospital that made use of Virtual reality therapy (VRT), the article shows that it is a form of technical reiteration designed to develop reflexivity about one's actions, provoking the craving so that it can be domesticated. The patient's engagement in "as if" mode is facilitated by the therapist's work in mapping the patient's addictive practices, personalizing the exposure situations, and elaborating the simulation as being part of a practical experiment. While the question of graphic realism does not seem especially problematic, the question of social plausibility can put the simulation operation under strain.

public, environmental & occupational health,social sciences, biomedical

VIF: Virtual Interactive Fiction (with a twist)

Interactive Fiction Games: A Colossal Adventure

Immersed in my Ideas: Using Virtual Reality and Multimodal Interactions to Visualize Users' Ideas and Thoughts

Physiologically Driven Storytelling: Concept and Software Tool

V-IRL: Grounding Virtual Intelligence in Real Life

Virtual Human: Storytelling and Computer Graphics for a Virtual Human Platform

Virtual reality as a tool for promoting reading via enhanced narrative absorption and empathy

iFiction: Mobile technology, new media, Mixed Reality and literary creativity in English teaching

EntangleVR++: Evaluating the Potential of using Entanglement in an Interactive VR Scene Creation System

Unrolling Virtual Worlds for Immersive Experiences

VRFromX: from Scanned Reality to Interactive Virtual Experience with Human-in-the-Loop.

Virtual Psychedelia

Design insights into embedding virtual reality content into life skills training for people with intellectual disability

Theoretical limits of Virtual Reality

Social Conjuring: Multi-User Runtime Collaboration with AI in Building Virtual 3D Worlds

What You Write Is What You Get: A Novel Mixed Reality Interface

The Character as Subjective Interface

Tangible Web: An Interactive Immersion Virtual RealityCreativity System that Travels Across Reality

Virtual Laboratories and Virtual Worlds

ConnectVR: A Trigger-Action Interface for Creating Agent-based Interactive VR Stories

Reality check. The issue of social plausibility in virtual reality therapy with addiction patients