Digital Humanistic Crowdsourcing of Gamification Based on Interactive Digital Narrative: Taking Henan Yuediao Opera as a Case

Zhenjia Fan,Han Li,Lijuan Yang

DOI: https://doi.org/10.1002/pra2.911

2023-10-23

Proceedings of the American Society for Information Science and Technology

Abstract:Proceedings of the Association for Information Science and Technology, Volume 60, Issue 1, Page 958-961, October 2023.

What problem does this paper attempt to address?

Exploitation Of Novel Multiplayer Gesture-Based Interaction And Virtual Puppetry For Digital Storytelling To Develop Children'S Narrative Skills

Hui Liang,Jian Chang,Shujie Deng,Can Chen,Ruofeng Tong,Jianjun Zhang

DOI: https://doi.org/10.1145/2817675.2817680

2015-01-01

Abstract:In recent years, digital storytelling has demonstrated powerful pedagogical functions by improving creativity, collaboration and intimacy among young children. Saturated with digital media technologies in their daily lives, the young generation demands natural interactive learning environments which offer multimodalities of feedback and meaningful immersive learning experiences. Virtual puppetry assisted storytelling system for young children, which utilises depth motion sensing technology and gesture control as the Human-Computer Interaction (HCI) method, has been proved to provide natural interactive learning experience for single player. In this paper, we designed and developed a novel system that allows multiple players to narrate, and most importantly, to interact with other characters and interactive virtual items in the virtual environment. We have conducted one user experiment with four young children for pedagogical evaluation and another user experiment with five postgraduate students for system evaluation. Our user study shows this novel digital storytelling system has great potential to stimulate learning abilities of young children through collaboration tasks.
Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Hui Liang,Jian Chang,Shujie Deng,Can Chen,Ruofeng Tong,Jian Jun Zhang

DOI: https://doi.org/10.1002/cav.1727

IF: 1.01

2016-01-01

Computer Animation and Virtual Worlds

Abstract:With the rapid development of human–computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head‐mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head‐mounted display outperformed the traditional monoscopic display in a comparison between the two.
Supporting Crowd Workers in Ideation Tasks Through Information Gathering and Reflective Activity

Lingyun Sun,Weiyue Gao,Wei Xiang

DOI: https://doi.org/10.1080/10447318.2022.2061748

IF: 4.92

2022-01-01

International Journal of Human-Computer Interaction

Abstract:Crowdsourcing is widely used to solve creative problems of ideation in HCI domain. To improve the creativity of crowdsourcing outcomes, researchers have proposed multiple approaches to support crowd workers' idea proposal step. Other steps in designers' creative process, such as information gathering and reflective activity, also impact idea creativity, while their effects in crowdsourcing scenarios remain unexplored. Therefore, referring to the creativity research, this study proposed an approach that optimized crowdsourcing tasks to help crowd workers perform information gathering and reflective activity. Two experiments involving 427 workers were conducted to test the effects of our approach. Results showed that instructing crowd workers to gather information and reflect on their ideas positively affected the creativity of crowdsourcing outcomes. This study provides inspiration for the optimization of crowdsourcing tasks and offers insights to the researchers who focus on using crowd power to achieve the ideation process.
Research on Intelligent Interaction of Digital Characters Based on Engine Technology : —take the Innovative Application of Mobile Terminal for Example

Li Huifang,Yin Jun

DOI: https://doi.org/10.1109/iciddt52279.2020.00018

2020-01-01

Abstract:Based on the application of engine technology and research on Innovative applications of mobile terminals, Intelligent digital characters are taken as the point of research in this paper to analyze the design process and conception thoughts of the intelligent behaviors through the engine technology platforms, so as to explore and summarize the experience and theories for constructing the intelligent behaviors of digital characters.
In-Situ Mode: Generative AI-Driven Characters Transforming Art Engagement Through Anthropomorphic Narratives

Yongming Li,Hangyue Zhang,Andrea Yaoyun Cui,Zisong Ma,Yunpeng Song,Zhongmin Cai,Yun Huang

2024-09-24

Abstract:Art appreciation serves as a crucial medium for emotional communication and sociocultural dialogue. In the digital era, fostering deep user engagement on online art appreciation platforms remains a challenge. Leveraging generative AI technologies, we present EyeSee, a system designed to engage users through anthropomorphic characters. We implemented and evaluated three modes (Narrator, Artist, and In-Situ) acting as a third-person narrator, a first-person creator, and first-person created objects, respectively, across two sessions: Narrative and Recommendation. We conducted a within-subject study with 24 participants. In the Narrative session, we found that the In-Situ and Artist modes had higher aesthetic appeal than the Narrator mode, although the Artist mode showed lower perceived usability. Additionally, from the Narrative to Recommendation session, we found that user-perceived relatability and believability within each interaction mode were sustained, but the user-perceived consistency and stereotypicality changed. Our findings suggest novel implications for applying anthropomorphic in-situ narratives to other educational settings.

Human-Computer Interaction
Innovative Digital Storytelling with AIGC: Exploration and Discussion of Recent Advances

Rongzhang Gu,Hui Li,Changyue Su,Wayne Wu

2023-09-28

Abstract:Digital storytelling, as an art form, has struggled with cost-quality balance. The emergence of AI-generated Content (AIGC) is considered as a potential solution for efficient digital storytelling production. However, the specific form, effects, and impacts of this fusion remain unclear, leaving the boundaries of AIGC combined with storytelling undefined. This work explores the current integration state of AIGC and digital storytelling, investigates the artistic value of their fusion in a sample project, and addresses common issues through interviews. Through our study, we conclude that AIGC, while proficient in image creation, voiceover production, and music composition, falls short of replacing humans due to the irreplaceable elements of human creativity and aesthetic sensibilities at present, especially in complex character animations, facial expressions, and sound effects. The research objective is to increase public awareness of the current state, limitations, and challenges arising from combining AIGC and digital storytelling.

Human-Computer Interaction,Artificial Intelligence,Computer Vision and Pattern Recognition,Graphics,Multimedia
Gamifying XAI: Enhancing AI Explainability for Non-technical Users through LLM-Powered Narrative Gamifications

Yuzhe You,Jian Zhao

2024-10-05

Abstract:Artificial intelligence (AI) has become tightly integrated into modern technology, yet existing exploratory visualizations for explainable AI (XAI) are primarily designed for users with technical expertise. This leaves everyday users, who also regularly interact with AI systems, with limited resources to explore or understand AI technologies they use. We propose a novel framework that enables non-technical users to collect insights by conversing directly with visualization elements via LLM-powered narrative gamifications. We implemented a prototype that utilizes such gamification to facilitate non-technical users' exploration of AI embedding projections. We conducted a comparative study with 10 participants to assess our prototype quantitatively and qualitatively. Our study results indicate that although our prototype effectively enhances non-technical users' AI/XAI knowledge, and users believe they learn more through the gamification feature, it remains inconclusive whether the gamification itself leads to further improvements in understanding. In addition, opinions among participants regarding the framework's engagement are mixed: some believe it enhances their exploration of the visualizations, while others feel it disrupts their workflow.

Human-Computer Interaction
Gamification of digital art: promoting speculative design and interactive experience

Jing Li

DOI: https://doi.org/10.1080/10494820.2022.2113099

2022-08-26

Interactive Learning Environments

Abstract:The development of new approaches to digital art is based on improving a variety of technologies. The main goal of this paper is to identify the gamification of digital art through the promotion of speculative design and interactive experiences. To achieve this goal, theoretical and practical methods were used in the research. A survey determined approaches the designers used to improve their skills. Most respondents (28%) prefer reading books and watching video blogs on their own while 22% of designers took professional courses. Therefore, mechanisms that promote respondents' involvement in selecting professional courses for designers have been described. Cognitive assessment of the skills acquired after completing the course showed significant advantages compared to those observed at the start of the experiment. This was confirmed by the Cohen coefficient. The hierarchy analysis method also contributed to the identification of program elements that are most important for the development of professional skills while not relying exclusively on the data from respondents. The value of the presented work lies in the possibility to improve the skills of gamification in the field of design with the help of the developed mechanisms, the effectiveness of which has been confirmed in practice.

education & educational research
Employing Gamified Crowdsourced Close-Range Sensing in the Pursuit of a Digital Twin of the Earth

Timo Nummenmaa,Samuli Laato,Philip Chambers,Tuomas Yrttimaa,Mikko Vastaranta,Oğuz Turan Buruk,Juho Hamari

DOI: https://doi.org/10.1080/10447318.2024.2352922

IF: 4.92

2024-06-06

International Journal of Human-Computer Interaction

Abstract:Advances in mobile consumer technology, especially in smartphones, have given rise to numerous new crowdsourcing opportunities. With the miniaturization of sensors and the growing number of smartphone models containing a wide range of them, we are now capable of creating digital imitations of real-world objects using everyday mobile devices. In this study, we created four augmented reality (AR) applications utilizing LiDAR sensors that were designed for crowdsourcing the creation of such digital imitations, i.e., digital twins. A user study was undertaken where video recordings, audio recordings, point cloud, and mesh data were collected in forest settings. Each approach was evaluated in terms of the characteristics and quality of the crowdsourced data acquired, as well as the participants' behavior and experience. Our findings demonstrate that through gamification approaches, we can influence not only the user experience but also the type and quality of the crowdsourced data. Our findings offer guidance on which gamification dynamics and interactions are best suited for several types of crowdsourcing tasks. We also show that the collection of meaningful data is not limited to specific interactions or experiences and suggest that there are multiple routes in gamification design for reaching the desired outcomes.

computer science, cybernetics,ergonomics
Gamification artifacts and crowdsourcing participation: Examining the mediating role of intrinsic motivations

Yuanyue Feng,Hua Jonathan Ye,Ying Yu,Congcong Yang,Tingru Cui

DOI: https://doi.org/10.1016/j.chb.2017.12.018

IF: 9.9

2018-04-01

Computers in Human Behavior

Abstract:Participation of individual workers (i.e., solvers) is critical to the viability and success of crowdsourcing platforms. Past literature indicates that gamification artifacts, by intriguing solvers’ intrinsic motivations, can encourage solvers’ participation in crowdsourcing. Nevertheless, little research has systemically theorized how intrinsic motivations mediate the relationship between gamification artifacts and crowdsourcing participation. Based on the motivational affordance perspective and related literature, this study theorizes gamification artifacts i.e., point rewarding and feedback giving, and identifies four intrinsic motivations (i.e., self-presentation, self-efficacy, social bonds, and playfulness) in the context of crowdsourcing. It then hypothesizes the mediating effects of the four intrinsic motivations on the relationships between the two gamification artifacts and crowdsourcing participation. It tests the model using survey data from 295 solvers in a large crowdsourcing platform. Results show that self-presentation, self-efficacy and playfulness positively mediates the impacts of two gamification artifacts on solvers’ participation. Theoretical contributions and practical implications are discussed.

psychology, multidisciplinary, experimental
Enabling Human-Centric Smart Cities: Crowdsourcing-Based Practice in China

Jiangtao Wang,Junfeng Zhao,Yong Zhang,Xin Peng,Ying Li,Yun Xie

DOI: https://doi.org/10.1109/mc.2018.2880016

2018-01-01

Computer

Abstract:Existing smart city systems are designed mainly for urban management from a governmental perspective, but they fail to cover the wide spectrum of citizens' daily lives. We propose a crowdsourcing-based platform through which multiple players can collaborate to produce more abundant, personalized, and proactive services.
Social Media Analytics of User Evaluation for Innovative Digital Cultural and Creative Products: Experiences Regarding Dunhuang Cultural Heritage

Ziyan Ai,Dickson K W Chiu,Kevin K W Ho

DOI: https://doi.org/10.1145/3653307

2024-03-26

Journal on Computing and Cultural Heritage

Abstract:Social media platforms play an increasingly important role in cultural communication as society develops, attracting promotions and discussions about digital cultural and creative products (CACPs). This research investigates the cultural collaboration between Tencent and Dunhuang Research Academy (Dunhuang Academy) and analyzes user evaluation of integrating cultural heritage education into CACPs. We obtained data through Weibo and compared user evaluations and semantic social network analysis of digital CACPs, including interactive products, games, and music. Results indicated that users were more interested in landscapes, dubbing, and user-generated content (UGC) for interactive products, character versions, posters and skills for games, and singers and songs for concerts. Semantic social network analysis was also used to explore the Dunhuang CACP Circle. Scant studies evaluate the usefulness of integrating cultural heritage into different digital CACPs, especially in Asia. Our suggestions help promoters understand user needs for digital CACPs and better user experience and value.

computer science, interdisciplinary applications
Gamification of Crowd-Driven Environment Design

Brandon Haworth,Muhammad Usman,Davide Schaumann,Nilay Chakraborty,Glen Berseth,Petros Faloutsos,Mubbasir Kapadia

DOI: https://doi.org/10.1109/mcg.2020.2965069

IF: 1.909

2021-07-01

IEEE Computer Graphics and Applications

Abstract:This article explores whether crowd-sourced human creativity within a gamified collaborative design framework can address the complexity of predictive environment design. This framework is predicated on gamifying crowd objectives and presenting environment design problems as puzzles. A usability study reveals that the framework is considered usable for the task. Participants were asked to configure an environment puzzle to reduce an important crowd metric, the total egress time. The design task was constructed to be straightforward and uses a simplified environment as a probe for understanding the utility of gamification and the performance of collaboration. Single-player and multiplayer designs outperformed both optimization and expert-sourced designs of the same environment and multiplayer designs further outperformed the single-player designs. Single-player and multiplayer iterations followed linear and exponential decrease trends in total egress time, respectively. Our experiments provide strong evidence toward an interesting novel approach of crowdsourcing collaborative environment design.

computer science, software engineering
Human–Computer Interaction and Participation in Software Crowdsourcing

Habib Ullah Khan,Farhad Ali,Yazeed Yasin Ghadi,Shah Nazir,Inam Ullah,Heba G. Mohamed

DOI: https://doi.org/10.3390/electronics12040934

IF: 2.9

2023-02-13

Electronics

Abstract:Improvements in communication and networking technologies have transformed people's lives and organizations' activities. Web 2.0 innovation has provided a variety of hybridized applications and tools that have changed enterprises' functional and communication processes. People use numerous platforms to broaden their social contacts, select items, execute duties, and learn new things. Context: Crowdsourcing is an internet-enabled problem-solving strategy that utilizes human–computer interaction to leverage the expertise of people to achieve business goals. In crowdsourcing approaches, three main entities work in collaboration to solve various problems. These entities are requestors (job providers), platforms, and online users. Tasks are announced by requestors on crowdsourcing platforms, and online users, after passing initial screening, are allowed to work on these tasks. Crowds participate to achieve various rewards. Motivation: Crowdsourcing is gaining importance as an alternate outsourcing approach in the software engineering industry. Crowdsourcing application development involves complicated tasks that vary considerably from the micro-tasks available on platforms such as Amazon Mechanical Turk. To obtain the tangible opportunities of crowdsourcing in the realm of software development, corporations should first grasp how this technique works, what problems occur, and what factors might influence community involvement and co-creation. Online communities have become more popular recently with the rise in crowdsourcing platforms. These communities concentrate on specific problems and help people with solving and managing these problems. Objectives: We set three main goals to research crowd interaction: (1) find the appropriate characteristics of social crowd utilized for effective software crowdsourcing, (2) highlight the motivation of a crowd for virtual tasks, and (3) evaluate primary participation reasons by assessing various crowds using Fuzzy AHP and TOPSIS method. Conclusion: We developed a decision support system to examine the appropriate reasons of crowd participation in crowdsourcing. Rewards and employments were evaluated as the primary motives of crowds for accomplishing tasks on crowdsourcing platforms, knowledge sharing was evaluated as the third reason, ranking was the fourth, competency was the fifth, socialization was sixth, and source of inspiration was the seventh.

engineering, electrical & electronic,computer science, information systems,physics, applied
Digital Storytelling of Intangible Cultural Heritage: A Multimodal Interactive Serious Game for Teaching Folk Dance

Yun Xie,Mingyang Su,Xiaomei Nie,Xiu Li

DOI: https://doi.org/10.1007/978-3-031-61147-6_22

2024-01-01

Abstract:As part of the intangible cultural heritage, folk dance is of great significance to the cultural identity of society. Digital technology is widely used to display tangible and intangible cultural heritage. Interactive digital storytelling become a normal way of museum exhibitions and virtual and augmented reality applications. This form has a good effect on digital assets of dance, to display background information, related events, and characters of cultural heritage objects. In this research, we design a folk dance teaching serious game based on multi-modal interaction. Through digital interactive narrative, players can learn folk dance movements and understand the cultural connotations behind the dance movements at the same time, to help folk dance get a better inheritance.
Designing Interactive Games based on Intangible Cultural Heritage: A Case Study on Chinese Nuo Culture

Zhuoer Wang,Jingya Li

DOI: https://doi.org/10.1145/3629606.3630201

2023-11-13

Abstract:The inheritance and promotion of Chinese intangible cultural heritage is an important way to strengthen cultural self-confidence and build a cultural power in the world. As the successors of Chinese cultures, young people must strengthen their cultural self-confidence and take the responsibilities in the construction of the cultural power. Digital games have been used to attract young people’s attention in intangible cultural heritage. In this paper, we gave designed and developed the “Nuo Game”, a puzzle game that combines both virtual content and physical materials, aimed at supporting young people to learn about the meaning of different masks in Nuo culture. The idea behind the game is to involve the player in solving puzzles in the virtual world by wearing different Nuo masks in the real world. In this paper, we depict the aesthetics, mechanics, game story, and technology used in the game based on the four-elements of game design. The integration of virtual and physical content with cultural stories embedded in the gameplay allow the player to learn the basics of Nuo culture in an engaging way.

Art,Computer Science,Education
From the main-melody story to gaming experience: is the Chinese digital game arena a potential counterpublic sphere?

Tingting Hu,Min Ouyang,Ziwei Zhang,Cathy Yue Wang

DOI: https://doi.org/10.1177/01634437221079562

2022-03-03

Media, Culture and Society

Abstract:This article interrogates the possibility that the digital game arena plays a role as a ‘counterpublic sphere’ where alternative voices that violate the hegemonic narratives in the mainstream media culture can be expressed. Exemplified by The Invisible Guardian, a Chinese interactive role-playing game, we present the first study to bring political perspective into the Chinese digital game studies, combining narrative analysis and players’ experience to provide a comprehensive understanding of political engagement in China’s digital game arena. We argue that the digital game uses various strategies to portray figures from different political parties in a way that subverts mainstream main-melody dramas and sheds light on sensitive historical movements. This implies the potential of Chinese digital games to become a counterpublic sphere, delivering alternative voices that challenge the mainstream media discourse and stimulate gamers’ political introspection.
Design and Analysis of a Collaborative Story Generation Game for Social Robots

Eric Nichols,Leo Gao,Yurii Vasylkiv,Randy Gomez

DOI: https://doi.org/10.3389/fcomp.2021.674333

2021-10-25

Frontiers in Computer Science

Abstract:Storytelling plays a central role in human socializing and entertainment, and research on conducting storytelling with robots is gaining interest. However, much of this research assumes that story content is curated. In this paper, we introduce the task of collaborative story generation , where an artificial intelligence agent, or a robot, and a person collaborate to create a unique story by taking turns adding to it. We present a collaborative story generation system which works with a human storyteller to create a story by generating new utterances based on the story so far. Our collaborative story generation system consists of a publicly-available large scale language model that was tuned on a dataset of writing prompts and short stories, and a ranker that samples from the language model and chooses the best possible output. We improve storytelling quality by optimizing the ranker’s sample size to strike a balance between quality and computational cost. Since latency can be detrimental to human-robot interaction, we examine the performance-latency trade-offs of our approach and find the optimal ranker sample size that strikes the best balance between quality and computational cost. We evaluate our system by having human participants play the collaborative story generation game and comparing the stories they create with our system to a naive baseline. Next, we conduct a detailed elicitation survey that sheds light on issues to consider when adapting our collaborative story generation system to a social robot. Finally, in a first step towards allowing human players to control the genre or mood of stories generated, we present preliminary work on steering story generation sentiment polarity with a sentiment analysis model. We find that our proposed method achieves a good balance of steering capability and text coherence. Our evaluation shows that participants have a positive view of collaborative story generation with a social robot and consider rich, emotive capabilities to be key to an enjoyable experience.
Digital Life Project: Autonomous 3D Characters with Social Intelligence

Zhongang Cai,Jianping Jiang,Zhongfei Qing,Xinying Guo,Mingyuan Zhang,Zhengyu Lin,Haiyi Mei,Chen Wei,Ruisi Wang,Wanqi Yin,Xiangyu Fan,Han Du,Liang Pan,Peng Gao,Zhitao Yang,Yang Gao,Jiaqi Li,Tianxiang Ren,Yukun Wei,Xiaogang Wang,Chen Change Loy,Lei Yang,Ziwei Liu

DOI: https://doi.org/10.48550/arXiv.2312.04547

2023-12-07

Computer Vision and Pattern Recognition

Abstract:In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment. Our framework comprises two primary components: 1) SocioMind: a meticulously crafted digital brain that models personalities with systematic few-shot exemplars, incorporates a reflection process based on psychology principles, and emulates autonomy by initiating dialogue topics; 2) MoMat-MoGen: a text-driven motion synthesis paradigm for controlling the character's digital body. It integrates motion matching, a proven industry technique to ensure motion quality, with cutting-edge advancements in motion generation for diversity. Extensive experiments demonstrate that each module achieves state-of-the-art performance in its respective domain. Collectively, they enable virtual characters to initiate and sustain dialogues autonomously, while evolving their socio-psychological states. Concurrently, these characters can perform contextually relevant bodily movements. Additionally, a motion captioning module further allows the virtual character to recognize and appropriately respond to human players' actions. Homepage: https://digital-life-project.com/

Digital Humanistic Crowdsourcing of Gamification Based on Interactive Digital Narrative: Taking Henan Yuediao Opera as a Case

Exploitation Of Novel Multiplayer Gesture-Based Interaction And Virtual Puppetry For Digital Storytelling To Develop Children'S Narrative Skills

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Supporting Crowd Workers in Ideation Tasks Through Information Gathering and Reflective Activity

Research on Intelligent Interaction of Digital Characters Based on Engine Technology : —take the Innovative Application of Mobile Terminal for Example

In-Situ Mode: Generative AI-Driven Characters Transforming Art Engagement Through Anthropomorphic Narratives

Innovative Digital Storytelling with AIGC: Exploration and Discussion of Recent Advances

Gamifying XAI: Enhancing AI Explainability for Non-technical Users through LLM-Powered Narrative Gamifications

Gamification of digital art: promoting speculative design and interactive experience

Employing Gamified Crowdsourced Close-Range Sensing in the Pursuit of a Digital Twin of the Earth

Gamification artifacts and crowdsourcing participation: Examining the mediating role of intrinsic motivations

Enabling Human-Centric Smart Cities: Crowdsourcing-Based Practice in China

Social Media Analytics of User Evaluation for Innovative Digital Cultural and Creative Products: Experiences Regarding Dunhuang Cultural Heritage

Gamification of Crowd-Driven Environment Design

Human–Computer Interaction and Participation in Software Crowdsourcing

Digital Storytelling of Intangible Cultural Heritage: A Multimodal Interactive Serious Game for Teaching Folk Dance

Designing Interactive Games based on Intangible Cultural Heritage: A Case Study on Chinese Nuo Culture

From the main-melody story to gaming experience: is the Chinese digital game arena a potential counterpublic sphere?

Design and Analysis of a Collaborative Story Generation Game for Social Robots

Digital Life Project: Autonomous 3D Characters with Social Intelligence