Tangible Music Interfaces Using Passive Magnetic Tags

Joseph A. Paradiso,Kai-yuh Hsiao,Ari Benbasat

DOI: https://doi.org/10.5281/zenodo.1176374

2020-10-04

Abstract:The technologies behind passive resonant magnetically coupled tags are introduced and their application as a musical controller is illustrated for solo or group performances, interactive installations, and music toys.

Human-Computer Interaction,Sound,Audio and Speech Processing

What problem does this paper attempt to address?

MagneChase: Create Chasing-Capturing Interactions Using Magnetic Potential Barrier for Tangible Games

Yuyang Zhang,Yue Yang,Lei Ren,Jie Wu,Jiaji Li,Chuyi Zhou,Yue Tao,Cheng Yang,Guanyun Wang,Ye Tao

DOI: https://doi.org/10.1145/3585088.3593900

2023-01-01

Abstract:"Like poles repel, unlike poles attract" is a fundamental principle of magnetism commonly used in instantaneous haptic interaction. Through the assembly design of basic five magnets, MagneChase creates a potential barrier between repulsion and attraction. This allows MagneChase modules to change the direction of their interacting force when brought closer. Applying this interaction principle, we provide application examples of kinetic roleplay to enhance tangible play experiences. A preliminary user study suggests that children were captivated by the magnetic phenomenon and derived pleasure from engaging with MagneChase. We also discuss the potential for MagneChase as a tangible kit to promote enlightenment learning in gameplay.
A tactile closed-loop device for musical interaction

Staas de Jong

DOI: https://doi.org/10.5281/zenodo.1176935

2021-07-27

Abstract:This paper presents a device implementing a closed tactile loop for musical interaction, based on a small freely held magnet which serves as the medium for both input and output. The component parts as well as an example of its programmable behaviour are described.

Human-Computer Interaction
Tangible Light: Back to Metaphor-Based Interaction

Yan Shi,Fangtian Ying,Jinhui Yu,Pengxiang Jia

DOI: https://doi.org/10.1109/iscid.2010.21

2010-01-01

Abstract:In this paper we present Tangible light, a simple tangible interface that use behavioral and emotional metaphor to convey meaning in its interaction design. Our purpose is to take advantage of natural physical metaphor to achieve a heightened legibility and seamlessness of interaction between people and information. At the same time we attempts to make control system invisible with embedded sensors. Advances in sensor technology have provided novel approach for user to interact with daily used technical device. Rather than issuing commands to a object via multi-purpose and abstract buttons the way is designed for more nuanced and application-specific approaches. So here we explore the coupling of emotion and action in an interface designed for metaphor and simplicity while manipulate lights as an instance.
Using Tangible Interaction to Design Musicking Artifacts for Non-musicians

Lucía Montesinos,Halfdan Hauch Jensen,Anders Sundnes Løvlie

2024-04-15

Abstract:This paper presents a Research through Design exploration of the potential for using tangible interactions to enable active music experiences - musicking - for non-musicians. We present the Tubularium prototype, which aims to help non-musicians play music without requiring any initial skill. We present the initial design of the prototype and the features implemented in order to enable music-making by non-musicians, and offer some reflections based on observations of informal initial user explorations of the prototype.

Human-Computer Interaction
Tongue–Computer Interface Prototype Design Based on T-Type Magnet Localization for Smart Environment Control

Hui-Min Shen,Yang Yue,Chong Lian,Di Ge,Geng Yang

DOI: https://doi.org/10.3390/app8122498

2018-01-01

Applied Sciences

Abstract:The interactions between paralyzed individuals with severe physical disabilities and smart infrastructure need to be facilitated, and the tongue–computer interface (TCI) provides an efficient and feasible solution. By attaching a permanent magnet (PM) on the apex of the tongue, the real-time tongue motion tracking can be switching to solve a nonlinear inverse magnetic problem. This paper presents a proof-of-concept prototype TCI system utilizing a combined T-type PM marker for potential environment control. The introduction of the combined T-type PM promotes the anisotropy of the magnetic field distribution. A comprehensive calibration method for the sensing system is proposed to figure out the bias in the magnetic moment of the PM marker and the sensing axis rotation of the sensors. To address the influence of initialization in solving the overdetermined inverse magnetic problem, an adaptive Levenberg–Marquardt algorithm is designed utilizing real-time measurements. Bench-top experiments were carried out based on a high-precision three-dimensional (3D) translation platform, and the feasibility of the proposed TCI system in magnetic localization accuracy and efficiency is fully assessed. The mean localization error is 1.65 mm with a mean processing time of 65.7 ms, and a mean improvement of 54.7% can be achieved compared with a traditional LM algorithm.
Musinger: Communication of Music over a Distance with Wearable Haptic Display and Touch Sensitive Surface

Miguel Altamirano Cabrera,Muhammad Haris Khan,Ali Alabbas,Luis Moreno,Issatay Tokmurziyev,Dzmitry Tsetserukou

2024-10-22

Abstract:This study explores the integration of auditory and tactile experiences in musical haptics, focusing on enhancing sensory dimensions of music through touch. Addressing the gap in translating auditory signals to meaningful tactile feedback, our research introduces a novel method involving a touch-sensitive recorder and a wearable haptic display that captures musical interactions via force sensors and converts these into tactile sensations. Previous studies have shown the potential of haptic feedback to enhance musical expressivity, yet challenges remain in conveying complex musical nuances. Our method aims to expand music accessibility for individuals with hearing impairments and deepen digital musical interactions. Experimental results reveal high accuracy ($98\%$ without noise, 93% with white noise) in melody recognition through tactile feedback, demonstrating effective transmission and perception of musical information. The findings highlight the potential of haptic technology to bridge sensory gaps, offering significant implications for music therapy, education, and remote musical collaboration, advancing the field of musical haptics and multi-sensory technology applications.

Human-Computer Interaction
MagPixel: Modular Toolkit for Designing Interactive Magnetic Shape Displays

Yitao Fan,Yifu Zhang,Boyu Feng,Deying Pan,Yuyang Zhang,Jiaji Li,Xinyi Liao,Xiaoliang Zhao,Ye Tao,Qi Wang,Lingyun Sun,Guanyun Wang

DOI: https://doi.org/10.1145/3613905.3650901

2024-01-01

Abstract:This paper introduces MagPixel, a modular toolkit for democratizing the design and fabrication of interactive shape displays using magnetic soft materials in the realm of Human-Computer Interaction (HCI). Known for their instant responsiveness and flexibility, magnetic soft materials have been used in material science and soft robotics but remain under-explored in HCI. MagPixel addresses this gap by offering a user-friendly, cost-effective solution for creating morphing tangible interfaces actuated by external magnetic fields. Existing methods of fabricating magnetic soft interfaces are complex, requiring multiple specialized tools and skills for casting, magnetization, and assembly. By simplifying fabrication into straightforward, standardized procedures, MagPixel eliminates the need for proprietary materials and extensive resources. The toolkit provides DIY enthusiasts in the maker community with the means to effortlessly create and modify interactive magnetic soft displays, MagPixel paves the way for broader experimentation and innovation in interactive design and HCI.
Body, Clothes, Water, and Toys: Media Towards Natural Music Expressions with Digital Sounds

Kenji Mase,Tomoko Yonezawa

DOI: https://doi.org/10.5281/zenodo.1176368

2020-10-04

Abstract:In this paper, we introduce our research challenges for creating new musical instruments using everyday-life media with intimate interfaces, such as the self-body, clothes, water and stuffed toys. Various sensor technologies including image processing and general touch sensitive devices are employed to exploit these interaction media. The focus of our effort is to provide user-friendly and enjoyable experiences for new music and sound performances. Multimodality of musical instruments is explored in each attempt. The degree of controllability in the performance and the richness of expressions are also discussed for each installation.

Human-Computer Interaction,Sound,Audio and Speech Processing
Tactile Melodies: A Desk-Mounted Haptics for Perceiving Musical Experiences

Raj Varshith Moora,Gowdham Prabhakar

2024-08-13

Abstract:This paper introduces a novel interface for experiencing music through haptic impulses to the palm of the hand. It presents a practical implementation of the system exploring the realm of musical haptics through the translation of MIDI data from a Digital Audio Workstation (DAW) into haptic sensations, from a set of haptic actuators, in real-time. It also includes a suitable music-to-haptic mapping strategy to translate notes from musical instruments to haptic feedback. The haptic actuators, placed strategically on the palmar surface of the hand allowed users to perceive music and were able to identify melody and rhythm of different musical compositions. A pilot user study conducted intended to assess the accuracy of the interface by testing the participants to select the correct audio presentation from the haptic presentation of the same musical composition. It presents a comparative study, differentiating between those with prior musical background and those without, in identifying the correct audio counterpart solely through haptic inputs. This pilot study delves into how users perceive and interpret haptic feedback within the context of musical compositions. The study showed promising results in enriching our understanding of user responses to haptic feedback in musical scenarios and exploring the intricacies of user experience with the system and its impact on musical interpretation.

Human-Computer Interaction
Bioinspired Magnetized String with Tension-Dependent Eigenfrequencies for Wearable Human-Machine Interactions

Biao Qi,Sen Ding,Yuanzhe Liang,Dan Fang,Ming Lei,Wenxue Dai,Chao Peng,Bingpu Zhou

DOI: https://doi.org/10.1021/acsami.4c16653

IF: 9.5

2024-11-28

ACS Applied Materials & Interfaces

Abstract:Flexible and wearable devices have exhibited potential for applications in the fields of human-machine interactions (HMIs) and Internet of Things. However, challenges remain in the improvement of the communication storage capacity with a simplified architecture. Inspired by tension regulation in natural tendons, a single-channel wearable HMI strategy is proposed using the eigenfrequency of magnetized strings as a sensing solution. Based on electromagnetic induction, mechanical vibration of the...

materials science, multidisciplinary,nanoscience & nanotechnology
A Touch on Musical Innovation: Exploring Wearables and Their Impact on New Interfaces for Musical Expression

David Wexler,Joanne Yip,Ka-Po Lee,Xiaolu Li,Yiu-Hong Wong

DOI: https://doi.org/10.3390/s24010250

2023-12-31

Abstract:This paper explores the innovative concept of using wearable technologies as a medium for musical expression. Special emphasis is placed on a unique wearable device equipped with motion, touch, and acceleration sensors, which can be used as a wrist strap, hand strap, or surface drum pad. The aim is to create a new musical instrument that simplifies music learning and expression and makes them more intuitive. The wearable device contains 32 individual touch-sensitive pressure sensors, a nine-axis inertial-measurement-unit motion sensor, and various light-emitting diode and vibrational haptic-feedback components. The inclusion of tactile and intuitive features in the wearable device enhances the musical experience of users by enabling engaging interaction. Consequently, it is believed that this groundbreaking technology has significant potential to contribute to the field of music, providing musicians with a versatile and intuitive instrument that facilitates their creative expression.
The cyclotactor: towards a tactile platform for musical interaction

Staas de Jong

DOI: https://doi.org/10.48550/arXiv.2107.12704

IF: 6.4588

2021-07-27

Human-Computer Interaction

Abstract:This paper reports on work in progress on a finger-based tactile I/O device for musical interaction. Central to the device is the ability to set up cyclical relationships between tactile input and output. A direct practical application of this to musical interaction is given, using the idea to multiplex two degrees of freedom on a single tactile loop.
MOONMENT: Designing Gesture-based Interaction with Acousto-Optic Feedbacks

Liang Tan,Jiayi Yao

DOI: https://doi.org/10.1145/3615522.3615564

2023-09-22

Abstract:This paper explores the interaction between human gesture and acousto-optic system through developing the interactive musical installation MOONMENT that outputs interactive music and light according to user's gestures. Based on interaction design principles and embodied metaphor theory, we present development process of the concept ideation, prototyping, and user testing. This work has various application potentials and can generate acousto-optic effects following the actions of singers, dancers, and actors. This study opens up new possibilities for future research in designing new interfaces of musical expression.

Computer Science,Art,Engineering
Dynamic Shaping of Multi-Touch Stimuli by Programmable Acoustic Metamaterial

Thomas Daunizeau,Sinan Haliyo,David Gueorguiev,Vincent Hayward

2024-08-19

Abstract:Acoustic metamaterials are artificial structures, often lattice of resonators, with unusual properties. They can be engineered to stop wave propagation in specific frequency bands. Once manufactured, their dispersive qualities remain invariant in time and space, limiting their practical use. Actively tuned arrangements have received growing interest to address this issue. Here, we introduce a new class of active metamaterial made from dual-state unit cells, either vibration sources when powered or passive resonators when left disconnected. They possess self-tuning capabilities, enabling deep subwavelength band gaps to automatically match the carrier signal of powered cells, typically around 200Hz. Swift electronic commutations between both states establish the basis for real-time reconfiguration of waveguides and shaping of vibration patterns. A series of experiments highlight how these tailored acceleration fields can spatially encode information relevant to human touch. This novel metamaterial can readily be made using off-the-shelf smartphone vibration motors, paving the way for a widespread adoption of multi-touch tactile displays.

Applied Physics,Human-Computer Interaction
Contactless Haptic Display Through Magnetic Field Control

Xiong Lu,Yuxing Yan,Beibei Qi,Huang Qian,Junbin Sun,Aaron Quigley

DOI: https://doi.org/10.48550/arXiv.2211.14163

IF: 6.4588

2022-11-25

Human-Computer Interaction

Abstract:Haptic rendering enables people to touch, perceive, and manipulate virtual objects in a virtual environment. Using six cascaded identical hollow disk electromagnets and a small permanent magnet attached to an operator's finger, this paper proposes and develops an untethered haptic interface through magnetic field control. The concentric hole inside the six cascaded electromagnets provides the workspace, where the 3D position of the permanent magnet is tracked with a Microsoft Kinect sensor. The driving currents of six cascaded electromagnets are calculated in real-time for generating the desired magnetic force. Offline data from an FEA (finite element analysis) based simulation, determines the relationship between the magnetic force, the driving currents, and the position of the permanent magnet. A set of experiments including the virtual object recognition experiment, the virtual surface identification experiment, and the user perception evaluation experiment were conducted to demonstrate the proposed system, where Microsoft HoloLens holographic glasses are used for visual rendering. The proposed magnetic haptic display leads to an untethered and non-contact interface for natural haptic rendering applications, which overcomes the constraints of mechanical linkages in tool-based traditional haptic devices.
Tele-touch embodied controllers: posthuman gestural interaction in music performance

Kirsty Beilharz

DOI: https://doi.org/10.1080/10350330.2011.591997

2011-09-01

Social Semiotics

Abstract:This article looks at the use of embodied music controllers for gestural interaction with sound. The technology for embodied interaction involves microchip “enhancement” of the human body, enabling the body as the music controller or musical actuator and connecting the performer with a networked (wireless) sonic performance environment. To examine the ways that the audience and performer respond to this new performance paradigm, this article first looks at some interpretations and understandings of posthumanisim and cyborgism and its effects for the music performer and audience. The influential body modifying and skin boundary cultures that colour audience perception and motivate cyborg processes, the body as expressive canvas and politicising cyborgism are considered in relation to alterity, community, marking and anthropology. The further social implications of radiofrequency identification tag technology and computing technologies that transfigure our body-limits and sense of self, privacy and human–machine integration are considered, particularly as contemporary posthuman tele-tactile technologies challenge conventional notions of touch and control. The Virtual Gamelan performance environment is presented as a test-bed for precision gestural interaction with sound, and to gauge audience feedback about these contentious contemporary issues that bleed into other pervasive technologies of the imminent future. This research was developed from the perspective of the music composer/performer and the motivation to explore posthuman cyborg notions through the embodiment of sensors in the disciplines of music, human–computer interaction, and design in a technological cyborg discussion that positions it differently to “mythological” cyborg discourse. The embodiment and permeation of the body are integral to the research goal of altering music performance paradigms. While utilising gesture, proprioception and spatial interaction, the embodiment of part of the system differentiates this approach from the Theremin or virtual interfaces with a physical controller (such as Guitar Hero or six-degrees-of-freedom devices with buttons and touch as well as motion, e.g. a Nintendo Wii controller) or the Xbox 360 Kinect, which uses low-latency life-like motion control of the on-screen avatar remote from the body of the performer.

communication,linguistics
Magnet Haptic: Using Magnets and Conductive Ink to Self-fabricate Interactive VR Devices

Cheng Yao,Shuyue Feng,Shichao Huang,Mengru Xue,Yuqi Hu,Haowen Ren,Fangtian Ying,Ran Wa

DOI: https://doi.org/10.1007/978-3-031-60012-8_28

2024-01-01

Abstract:Magnet Haptic method presents an innovative approach for quickly creating tactile input devices tailored to enhance multisensory experiences in virtual reality (VR) environments. This technique utilizes a unique combination of magnets and conductive ink, allowing users to design and fabricate tactile feedback mechanisms easily. By exploiting the magnetism of the magnets to form various tactile structures and integrating the conductive properties of the ink for signal transmission, this method seamlessly bridges the tactile interactionwithVR devices. A prototype was developed and evaluated with the participation of eight users, who engaged in both the creation and usage of the tactile devices. Their experiences, gathered through semi-structured interviews, provided insights into the usability and versatility of the method. Additionally, the durability and performance of the hardware were rigorously tested in multi-layer configurations and through repeated use, demonstrating its potential for diverse and complex VR applications. This research contributes significantly to the field by offering a fast, customizable solution for developing tactile interfaces in VR, marking a step forward in achieving comprehensive multisensory virtual experiences.
A Soft Tactile Sensor Based on Magnetics and Hybrid Flexible-Rigid Electronics

Miguel Neto,Pedro Ribeiro,Ricardo Nunes,Lorenzo Jamone,Alexandre Bernardino,Susana Cardoso

DOI: https://doi.org/10.3390/s21155098

IF: 3.9

2021-07-28

Sensors

Abstract:Tactile sensing is crucial for robots to manipulate objects successfully. However, integrating tactile sensors into robotic hands is still challenging, mainly due to the need to cover small multi-curved surfaces with several components that must be miniaturized. In this paper, we report the design of a novel magnetic-based tactile sensor to be integrated into the robotic hand of the humanoid robot Vizzy. We designed and fabricated a flexible 4 × 2 matrix of Si chips of magnetoresistive spin valve sensors that, coupled with a single small magnet, can measure contact forces from 0.1 to 5 N on multiple locations over the surface of a robotic fingertip; this design is innovative with respect to previous works in the literature, and it is made possible by careful engineering and miniaturization of the custom-made electronic components that we employ. In addition, we characterize the behavior of the sensor through a COMSOL simulation, which can be used to generate optimized designs for sensors with different geometries.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Design Methodology for Magnetic Field-Based Soft Tri-Axis Tactile Sensors

Hongbo Wang,Greg de Boer,Junwai Kow,Ali Alazmani,Mazdak Ghajari,Robert Hewson,Peter Culmer

DOI: https://doi.org/10.3390/s16091356

IF: 3.9

2016-08-24

Sensors

Abstract:Tactile sensors are essential if robots are to safely interact with the external world and to dexterously manipulate objects. Current tactile sensors have limitations restricting their use, notably being too fragile or having limited performance. Magnetic field-based soft tactile sensors offer a potential improvement, being durable, low cost, accurate and high bandwidth, but they are relatively undeveloped because of the complexities involved in design and calibration. This paper presents a general design methodology for magnetic field-based three-axis soft tactile sensors, enabling researchers to easily develop specific tactile sensors for a variety of applications. All aspects (design, fabrication, calibration and evaluation) of the development of tri-axis soft tactile sensors are presented and discussed. A moving least square approach is used to decouple and convert the magnetic field signal to force output to eliminate non-linearity and cross-talk effects. A case study of a tactile sensor prototype, MagOne, was developed. This achieved a resolution of 1.42 mN in normal force measurement (0.71 mN in shear force), good output repeatability and has a maximum hysteresis error of 3.4%. These results outperform comparable sensors reported previously, highlighting the efficacy of our methodology for sensor design.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Soft Magnetoelastic Tactile Multi-Sensors with Energy-Absorbing Properties for Self-Powered Human-Machine Interfaces

Liqiong Lin,Jianyou Zhou,Zheng Zhong

DOI: https://doi.org/10.1021/acsami.4c10703

IF: 9.5

2024-09-13

ACS Applied Materials & Interfaces

Abstract:Tactile sensors play a key role in human-machine interfaces (HMIs) for augmented and virtual reality, point-of-care devices, and human-robot collaboration, which show the promise of revolutionizing our ways of life. Here, we present a sensor (EMTS) that utilizes the magnetoelastic effect in a soft metamaterial to convert mechanical pressure into electrical signals. With this unique mechanism, the proposed EMTS simultaneously possesses self-powering, waterproof, and compliant features. The soft...

materials science, multidisciplinary,nanoscience & nanotechnology

Tangible Music Interfaces Using Passive Magnetic Tags

MagneChase: Create Chasing-Capturing Interactions Using Magnetic Potential Barrier for Tangible Games

A tactile closed-loop device for musical interaction

Tangible Light: Back to Metaphor-Based Interaction

Using Tangible Interaction to Design Musicking Artifacts for Non-musicians

Tongue–Computer Interface Prototype Design Based on T-Type Magnet Localization for Smart Environment Control

Musinger: Communication of Music over a Distance with Wearable Haptic Display and Touch Sensitive Surface

MagPixel: Modular Toolkit for Designing Interactive Magnetic Shape Displays

Body, Clothes, Water, and Toys: Media Towards Natural Music Expressions with Digital Sounds

Tactile Melodies: A Desk-Mounted Haptics for Perceiving Musical Experiences

Bioinspired Magnetized String with Tension-Dependent Eigenfrequencies for Wearable Human-Machine Interactions

A Touch on Musical Innovation: Exploring Wearables and Their Impact on New Interfaces for Musical Expression

The cyclotactor: towards a tactile platform for musical interaction

MOONMENT: Designing Gesture-based Interaction with Acousto-Optic Feedbacks

Dynamic Shaping of Multi-Touch Stimuli by Programmable Acoustic Metamaterial

Contactless Haptic Display Through Magnetic Field Control

Tele-touch embodied controllers: posthuman gestural interaction in music performance

Magnet Haptic: Using Magnets and Conductive Ink to Self-fabricate Interactive VR Devices

A Soft Tactile Sensor Based on Magnetics and Hybrid Flexible-Rigid Electronics

Design Methodology for Magnetic Field-Based Soft Tri-Axis Tactile Sensors

Soft Magnetoelastic Tactile Multi-Sensors with Energy-Absorbing Properties for Self-Powered Human-Machine Interfaces