Abstract:Taiwan’s heritage in terms of the local music culture has been gradually fading in recent years. Thus, boosting and passing on the local music culture to pre-school and elementary school students are urgent requirements. This study proposes an interactive integration of multi-sensory and volumetric content for music education in Taiwan into applications for children. Further, the study introduces a technological multi-sensory pop-up sketch book created in collaboration with the National Taiwan Symphony Orchestra (NTSO) and Industrial Technology Research Institute (ITRI). Both organizations collaborate to integrate emerging media technologies, including augmented reality (AR) and volumetric capture for content production, and creative music teaching methods, derived from traditional pop-up sketch books. The final product featured 3D animated videos to achieve interactive learning with digital formation additional to the real worlds. This AR multi-sensory pop-up sketch book utilizes advanced volumetric capture technology to capture the motions of main actors in vivid 3D animation. Besides, modularized pop-up cards of musical instruments provide a haptic experience to complement the story. The book targets children aged between 3 and 12 years. An actual reading survey was conducted on 497 students from five kindergartens in Taiwan. Satisfaction with the book was rated using a five-rank scale. The cluster random sampling method was used for data analysis. Results of t-test produced an average score of 4.9980, which indicated that the target audience ranked the book with high levels of satisfaction. The results also confirmed that the story line was familiar among young children within the target age range. Additionally, using digital audio–visual augmented reality technology will be conducive to young children in terms of acceptance and recognition of traditional music culture. 3D animations of famous intellectual property characters aroused the interest of readers in learning about music through interactive contents. Moreover, this study provided evidence that the STEAM education model, which represents a cross-curricular approach that integrates Science, Technology, Engineering, Art, and Mathematics, is applicable to the inheritance and development of the local music culture.

An Interactive Smart Music Toy Design for Children

DrumGenius: bridging learning-gap with interactive musical instruments.

Interactive multi-sensory and volumetric content integration for music education applications

Research on the design of intelligent flowerpot science and education toys

Smart Toys Design Opportunities for Measuring Children's Fine Motor Skills Development

User Experience Research of Toy Design

A Study on the Interactive "HOPSCOTCH" Game for the Children Using Computer Music Techniques

Tinkling: Research on the design of a Bianqing interactive system for children based on experiential learning

The Design of a Novel Digital Puzzle Gaming System for Young Children’s Learning by Interactive Multi-Sensing and Tangible User Interfacing Techniques

Research on Intelligent Design of Children's Module Toys

Design music games based on the play of folk games to create excitement for preschool children 5 – 6 years old

FunEat: an Interactive Tableware for Improving Eating Habits in Children

Tangible Interaction Design in Music Intervention for Chinese Children with Developmental Dyslexia

Design For Learning Through Play. An Exploratory Study On Chinese Perspective

MusicCollage: A Music Composition Tool for Children Based on Synesthesia and a Genetic Algorithm

Students’ Attitudes and Competences in Modeling Using 3D Cartoon Toy Design Maker

A Tangible Jigsaw Puzzle Prototype for Attention-Deficit Hyperactivity Disorder Children

Interactive music video games and children's musical development

The design of children's educational toys

Design and Evaluation of Children's Assembled Toys Based on Mortise and Tenon Structure and Zodiac Modeling

SoundTag: children's interactive play based upon RFID employed wearable computer