Abstract:Motion capture is a technique of digitally recording the movements of real entities, usually humans. It was originally developed as an analysis tool in biomechanics research, but has grown increasingly important as a source of motion data for computer animation. In this context it has been widely used for both cinema and video games. Hand motion capture and tracking in particular has received a lot of attention because of its critical role in the design of new Human Computer Interaction methods and gesture analysis. One of the main difficulties is the capture of human hand motion. This paper gives an overview of ongoing research “HandPuppet3D” being carried out in collaboration with an animation studio to employ computer vision techniques to develop a prototype desktop system and associated animation process that will allow an animator to control 3D character animation through the use of hand gestures. The eventual goal of the project is to support existing practice by providing a softer, more intuitive, user interface for the animator that improves the productivity of the animation workflow and the quality of the resulting animations. To help achieve this goal the focus has been placed on developing a prototype camera based desktop gesture capture system to capture hand gestures and interpret them in order to generate and control the animation of 3D character models. This will allow an animator to control 3D character animation through the capture and interpretation of hand gestures. Methods will be discussed for motion tracking and capture in 3D animation and in particular that of hand motion tracking and capture. HandPuppet3D aims to enable gesture capture with interpretation of the captured gestures and control of the target 3D animation software. This involves development and testing of a motion analysis system built from algorithms recently developed. We review current software and research methods available in this area and describe our current work.

A creation-tool for contemporary dance using multimodal video annotation

Annotating the captured dance: reflections on the role of tool-creation

A Framework of Mocap Data Editing with Labanotation

Interactive Dance Choreography Assistance

Automatic Composition by Body-Part Motion Synthesis for Supporting Dance Creation

Implementation of a notation-based motion choreography system

An Approach to the Creation and Presentation of Reference Gesture Datasets, for the Preservation of Traditional Crafts

Mocap data editing via movement notations

Digital dance writing. A teaching tool to support the understanding of Digital Choreography

Virtual Character Animation Based on Affordable Motion Capture and Reconfigurable Tangible Interfaces

Exploring AI-assisted Ideation and Prototyping for Choreography

Exploration of direct bi-manual interaction in digitally mediated stop-motion animation

VideoDoodles: Hand-Drawn Animations on Videos with Scene-Aware Canvases

DanceGen: Supporting Choreography Ideation and Prototyping with Generative AI

ExpressEdit: Video Editing with Natural Language and Sketching

Choreographing the Digital Canvas: A Machine Learning Approach to Artistic Performance

Visual instrument co-design embracing the unique movement capabilities of a dancer with physical disability

DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis

Towards an interactive multimedia experience for club music and dance

HandPuppet3D: Motion capture and analysis for character animation

ChoreoVis: Planning and Assessing Formations in Dance Choreographies