Abstract:The issue of how a robot playing instruments is one of the core challenges in the area of robotic musician and has been studied for many years. Most previous research mainly relied either on the close-loop adjusting manually or on the predesigned characteristic function with the expertise, while ignoring to learn the mechanism of human in some ways. Enlightened by recent works (humanoid walking, robot fall protection, object grasping, etc.) in robotics community, where inspirations from human being offer a promising alternative in robots acquiring their skills, the issue of how a robot learning the human-inspired drum-playing skill is addressed in this research. With the investigation on the learning process of human, a new approach for robots autonomously developing their drum-playing skill is proposed. Firstly, the robot developmentally learns its internal model through repeated sensorimotor experiences between its motor system and the sounds. Then it constantly adjusts its action according to auditory feedback under an open-ended learning style. Experimental results show that the proposed approach is effective for a robot to achieve the drum-playing skill autonomously. The robot can predict the likely sound of a planned or executed motor action with an average similarity of 81.19%, and the motor action required to achieve a desired sound with an average similarity of 81.28%. Furthermore, with our open-ended algorithm, the robot can achieve a desired sound with an average similarity of 96.93% via average 1.16 times adjustment. After adjusting, the internal model will be updated according to the sensorimotor experiences produced in previous adjustment process. Utilizing the new model, the robot can predict the motor action required to achieve a desired sound with an average similarity of 97.46%.

Developing Robot Drumming Skill with Listening-Playing Loop.

Robot Learning to Play Drums with an Open-Ended Internal Model

Performance Control System of Dulcimer Music-Playing Robot

DrumGenius: bridging learning-gap with interactive musical instruments.

Improving Drumming Robot Via Attention Transformer Network

DeepDrummer : Generating Drum Loops using Deep Learning and a Human in the Loop

Robot Learning from Human Demonstration for Playing Ocarina

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Contactless Interaction System Based on Facial Expression Recognition for Humanoid Piano Robot

Dance To Music Expressively: A Brain-Inspired System Based On Audio-Semantic Model For Cognitive Development Of Robots

Designing a Musical Robot for Chinese Bamboo Flute Performance

Human-Robot Cooperative Piano Playing with Learning-Based Real-Time Music Accompaniment

That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulation

Learning to Communicate Functional States with Nonverbal Expressions for Improved Human-Robot Collaboration

Voice coding interactive music game teaching mode based on wireless network

Demonstration Learning and Generalization of Robotic Motor Skills Based on Wearable Motion Tracking Sensors

ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data

AIoT-Based Drum Transcription Robot using Convolutional Neural Networks

A survey of robot learning from demonstrations for Human-Robot Collaboration

From Rolling Over to Walking: Enabling Humanoid Robots to Develop Complex Motor Skills

Robot Skill Learning in Human-Robot Collaboration with Dynamic Time Warping Iterative Learning Control