A tactile closed-loop device for musical interaction

Staas de Jong

DOI: https://doi.org/10.5281/zenodo.1176935

2021-07-27

Abstract:This paper presents a device implementing a closed tactile loop for musical interaction, based on a small freely held magnet which serves as the medium for both input and output. The component parts as well as an example of its programmable behaviour are described.

Human-Computer Interaction

What problem does this paper attempt to address?

The cyclotactor: towards a tactile platform for musical interaction

Staas de Jong

DOI: https://doi.org/10.48550/arXiv.2107.12704

IF: 6.4588

2021-07-27

Human-Computer Interaction

Abstract:This paper reports on work in progress on a finger-based tactile I/O device for musical interaction. Central to the device is the ability to set up cyclical relationships between tactile input and output. A direct practical application of this to musical interaction is given, using the idea to multiplex two degrees of freedom on a single tactile loop.
Tangible Music Interfaces Using Passive Magnetic Tags

Joseph A. Paradiso,Kai-yuh Hsiao,Ari Benbasat

DOI: https://doi.org/10.5281/zenodo.1176374

2020-10-04

Abstract:The technologies behind passive resonant magnetically coupled tags are introduced and their application as a musical controller is illustrated for solo or group performances, interactive installations, and music toys.

Human-Computer Interaction,Sound,Audio and Speech Processing
Developing the cyclotactor

Staas de Jong

DOI: https://doi.org/10.5281/zenodo.1177591

2021-07-27

Abstract:This paper presents developments in the technology underlying the cyclotactor, a finger-based tactile I/O device for musical interaction. These include significant improvements both in the basic characteristics of tactile interaction and in the related (vibro)tactile sample rates, latencies, and timing precision. After presenting the new prototype's tactile output force landscape, some of the new possibilities for interaction are discussed, especially those for musical interaction with zero audio/tactile latency.

Human-Computer Interaction
A Magnet‐Driven Soft Bistable Actuator

Zhou Chen,Shangcheng Kong,Yunhu He,Shiting Chen,Wanying Wang,Lihan Jin,Shun Zhang,Ying Hong,Lulu Pan,Haikun Wu,Youneng Xie,Changhong Linghu,Zhengyi Mao,Zhengbao Yang,Chi Hou Chan,Jizhou Song,Jian Lu

DOI: https://doi.org/10.1002/adfm.202311498

IF: 19

2024-01-01

Advanced Functional Materials

Abstract:Bistable morphing structures are widely used as actuation mechanisms in soft actuators, soft robotics, energy absorbers, mechanical computers, optical lenses, metamaterials, and flexible electronics. However, untethered actuators, repetitive actuators, and hybrid-assembly (containing in-plane-assembly and out-of-plane-assembly) actuators remain challenging to realize using existing bistable structures, which hinders the novel application of such actuators in research, engineering, and daily life. This problem is solved by fabricating a magnet-driven soft bistable actuator (MSBA) unit. The self-buckling of the circular polydimethylsiloxane (PDMS) sheet ensures the bistability of the actuator and allows it to operate as an independent unit, free of external constraints. The reorientation of neodymium-iron-boron (NdFeB) microparticles embedded in the PDMS sheet enables the dome-shaped actuators to exhibit repetitive snapping under the stimulus of a direction-switching magnetic field. The potential of this MSBA unit in bionics, electronics, and biomechanics applications is demonstrated in systematic studies involving modification of the buckling deflection and magnetic moment density. The MSBA unit exhibits excellent performance in hybrid designs and intelligent systems. This work develops a magnet-driven soft bistable actuator embedded with reoriented hard magnetic microparticles. This protocol realizes untethered and reversible bistable behavior with a switch in the direction of the magnetic field. The magnet-driven soft bistable actuator (MSBA) can be assembled in-plane, out-of-plane, or in a hybrid manner, offering multistable configurations and finding its potential application in biomimetic grippers, reconfigurable electronics, and dynamic bioreactor.image
Generating Clear Vibrotactile Cues with a Magnet Embedded in a Soft Finger Sheath

Ifat Gertler,Gokhan Serhat,Katherine J Kuchenbecker

DOI: https://doi.org/10.1089/soro.2021.0184

Abstract:Haptic displays act on the user's body to stimulate the sense of touch and enrich applications from gaming and computer-aided design to rehabilitation and remote surgery. However, when crafted from typical rigid robotic components, they tend to be heavy, bulky, and expensive, while sleeker designs often struggle to create clear haptic cues. This article introduces a lightweight wearable silicone finger sheath that can deliver salient and rich vibrotactile cues using electromagnetic actuation. We fabricate the sheath on a ferromagnetic mandrel with a process based on dip molding, a robust fabrication method that is rarely used in soft robotics but is suitable for commercial production. A miniature rare-earth magnet embedded within the silicone layers at the center of the finger pad is driven to vibrate by the application of alternating current to a nearby air-coil. Experiments are conducted to determine the amplitude of the magnetic force and the frequency response function for the displacement amplitude of the magnet perpendicular to the skin. In addition, high-fidelity finite element analyses of the finger wearing the device are performed to investigate the trends observed in the measurements. The experimental and simulated results show consistent dynamic behavior from 10 to 1000 Hz, with the displacement decreasing after about 300 Hz. These results match the detection threshold profile obtained in a psychophysical study performed by 17 users, where more current was needed only at the highest frequency. A cue identification experiment and a demonstration in virtual reality validate the feasibility of this approach to fingertip haptics.
A Touch on Musical Innovation: Exploring Wearables and Their Impact on New Interfaces for Musical Expression

David Wexler,Joanne Yip,Ka-Po Lee,Xiaolu Li,Yiu-Hong Wong

DOI: https://doi.org/10.3390/s24010250

2023-12-31

Abstract:This paper explores the innovative concept of using wearable technologies as a medium for musical expression. Special emphasis is placed on a unique wearable device equipped with motion, touch, and acceleration sensors, which can be used as a wrist strap, hand strap, or surface drum pad. The aim is to create a new musical instrument that simplifies music learning and expression and makes them more intuitive. The wearable device contains 32 individual touch-sensitive pressure sensors, a nine-axis inertial-measurement-unit motion sensor, and various light-emitting diode and vibrational haptic-feedback components. The inclusion of tactile and intuitive features in the wearable device enhances the musical experience of users by enabling engaging interaction. Consequently, it is believed that this groundbreaking technology has significant potential to contribute to the field of music, providing musicians with a versatile and intuitive instrument that facilitates their creative expression.
A Soft Tactile Sensor Based on Magnetics and Hybrid Flexible-Rigid Electronics

Miguel Neto,Pedro Ribeiro,Ricardo Nunes,Lorenzo Jamone,Alexandre Bernardino,Susana Cardoso

DOI: https://doi.org/10.3390/s21155098

IF: 3.9

2021-07-28

Sensors

Abstract:Tactile sensing is crucial for robots to manipulate objects successfully. However, integrating tactile sensors into robotic hands is still challenging, mainly due to the need to cover small multi-curved surfaces with several components that must be miniaturized. In this paper, we report the design of a novel magnetic-based tactile sensor to be integrated into the robotic hand of the humanoid robot Vizzy. We designed and fabricated a flexible 4 × 2 matrix of Si chips of magnetoresistive spin valve sensors that, coupled with a single small magnet, can measure contact forces from 0.1 to 5 N on multiple locations over the surface of a robotic fingertip; this design is innovative with respect to previous works in the literature, and it is made possible by careful engineering and miniaturization of the custom-made electronic components that we employ. In addition, we characterize the behavior of the sensor through a COMSOL simulation, which can be used to generate optimized designs for sensors with different geometries.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Tactile Melodies: A Desk-Mounted Haptics for Perceiving Musical Experiences

Raj Varshith Moora,Gowdham Prabhakar

2024-08-13

Abstract:This paper introduces a novel interface for experiencing music through haptic impulses to the palm of the hand. It presents a practical implementation of the system exploring the realm of musical haptics through the translation of MIDI data from a Digital Audio Workstation (DAW) into haptic sensations, from a set of haptic actuators, in real-time. It also includes a suitable music-to-haptic mapping strategy to translate notes from musical instruments to haptic feedback. The haptic actuators, placed strategically on the palmar surface of the hand allowed users to perceive music and were able to identify melody and rhythm of different musical compositions. A pilot user study conducted intended to assess the accuracy of the interface by testing the participants to select the correct audio presentation from the haptic presentation of the same musical composition. It presents a comparative study, differentiating between those with prior musical background and those without, in identifying the correct audio counterpart solely through haptic inputs. This pilot study delves into how users perceive and interpret haptic feedback within the context of musical compositions. The study showed promising results in enriching our understanding of user responses to haptic feedback in musical scenarios and exploring the intricacies of user experience with the system and its impact on musical interpretation.

Human-Computer Interaction
A Mechatronic Platform for Studies of Tactile Sensory System on Prosthesis Hand

Xiaoying Cheng,Qingchuan Shan,Pin Yu,Liling Cheng,Wei-ting Liu,Xin Fu

DOI: https://doi.org/10.1007/978-3-642-40852-6_51

2013-01-01

Abstract:This paper focuses on developing a mechatronic platform for smart prosthesis touch studies of tactile sensory system. The platform was designed for stimulation of the tactile sensor arrays using multiple indenters driven by an auto positioner, providing repeatable stimuli with controlled force. Particular requirements were addressed to make the platform suitable for curved sensitive flat surface on fingertip of smart prosthesis hand. The design of the mechatronic platform is presented, as well as its performance in stimulating process.
A Virtual 2D Tactile Array for Soft Actuators Using Acoustic Sensing

Vincent Wall,Oliver Brock

DOI: https://doi.org/10.48550/arXiv.2208.10218

2022-08-22

Abstract:We create a virtual 2D tactile array for soft pneumatic actuators using embedded audio components. We detect contact-specific changes in sound modulation to infer tactile information. We evaluate different sound representations and learning methods to detect even small contact variations. We demonstrate the acoustic tactile sensor array by the example of a PneuFlex actuator and use a Braille display to individually control the contact of 29x4 pins with the actuator's 90x10 mm palmar surface. Evaluating the spatial resolution, the acoustic sensor localizes edges in x- and y-direction with a root-mean-square regression error of 1.67 mm and 0.0 mm, respectively. Even light contacts of a single Braille pin with a lifting force of 0.17 N are measured with high accuracy. Finally, we demonstrate the sensor's sensitivity to complex contact shapes by successfully reading the 26 letters of the Braille alphabet from a single display cell with a classification rate of 88%.

Robotics
Magnet Haptic: Using Magnets and Conductive Ink to Self-fabricate Interactive VR Devices

Cheng Yao,Shuyue Feng,Shichao Huang,Mengru Xue,Yuqi Hu,Haowen Ren,Fangtian Ying,Ran Wa

DOI: https://doi.org/10.1007/978-3-031-60012-8_28

2024-01-01

Abstract:Magnet Haptic method presents an innovative approach for quickly creating tactile input devices tailored to enhance multisensory experiences in virtual reality (VR) environments. This technique utilizes a unique combination of magnets and conductive ink, allowing users to design and fabricate tactile feedback mechanisms easily. By exploiting the magnetism of the magnets to form various tactile structures and integrating the conductive properties of the ink for signal transmission, this method seamlessly bridges the tactile interactionwithVR devices. A prototype was developed and evaluated with the participation of eight users, who engaged in both the creation and usage of the tactile devices. Their experiences, gathered through semi-structured interviews, provided insights into the usability and versatility of the method. Additionally, the durability and performance of the hardware were rigorously tested in multi-layer configurations and through repeated use, demonstrating its potential for diverse and complex VR applications. This research contributes significantly to the field by offering a fast, customizable solution for developing tactile interfaces in VR, marking a step forward in achieving comprehensive multisensory virtual experiences.
Musinger: Communication of Music over a Distance with Wearable Haptic Display and Touch Sensitive Surface

Miguel Altamirano Cabrera,Muhammad Haris Khan,Ali Alabbas,Luis Moreno,Issatay Tokmurziyev,Dzmitry Tsetserukou

2024-10-22

Abstract:This study explores the integration of auditory and tactile experiences in musical haptics, focusing on enhancing sensory dimensions of music through touch. Addressing the gap in translating auditory signals to meaningful tactile feedback, our research introduces a novel method involving a touch-sensitive recorder and a wearable haptic display that captures musical interactions via force sensors and converts these into tactile sensations. Previous studies have shown the potential of haptic feedback to enhance musical expressivity, yet challenges remain in conveying complex musical nuances. Our method aims to expand music accessibility for individuals with hearing impairments and deepen digital musical interactions. Experimental results reveal high accuracy ($98\%$ without noise, 93% with white noise) in melody recognition through tactile feedback, demonstrating effective transmission and perception of musical information. The findings highlight the potential of haptic technology to bridge sensory gaps, offering significant implications for music therapy, education, and remote musical collaboration, advancing the field of musical haptics and multi-sensory technology applications.

Human-Computer Interaction
Using Tangible Interaction to Design Musicking Artifacts for Non-musicians

Lucía Montesinos,Halfdan Hauch Jensen,Anders Sundnes Løvlie

2024-04-15

Abstract:This paper presents a Research through Design exploration of the potential for using tangible interactions to enable active music experiences - musicking - for non-musicians. We present the Tubularium prototype, which aims to help non-musicians play music without requiring any initial skill. We present the initial design of the prototype and the features implemented in order to enable music-making by non-musicians, and offer some reflections based on observations of informal initial user explorations of the prototype.

Human-Computer Interaction
Handheld Haptic Device with Coupled Bidirectional Input

Megh Vipul Doshi,Michael Hagenow,Robert Radwin,Michael Gleicher,Bilge Mutlu,Michael Zinn

DOI: https://doi.org/10.1109/WHC56415.2023.10224398

2023-05-31

Abstract:Handheld kinesthetic haptic interfaces can provide greater mobility and richer tactile information as compared to traditional grounded devices. In this paper, we introduce a new handheld haptic interface which takes input using bidirectional coupled finger flexion. We present the device design motivation and design details and experimentally evaluate its performance in terms of transparency and rendering bandwidth using a handheld prototype device. In addition, we assess the device's functional performance through a user study comparing the proposed device to a commonly used grounded input device in a set of targeting and tracking tasks.

Systems and Control,Human-Computer Interaction,Robotics
Design Methodology for Magnetic Field-Based Soft Tri-Axis Tactile Sensors

Hongbo Wang,Greg de Boer,Junwai Kow,Ali Alazmani,Mazdak Ghajari,Robert Hewson,Peter Culmer

DOI: https://doi.org/10.3390/s16091356

IF: 3.9

2016-08-24

Sensors

Abstract:Tactile sensors are essential if robots are to safely interact with the external world and to dexterously manipulate objects. Current tactile sensors have limitations restricting their use, notably being too fragile or having limited performance. Magnetic field-based soft tactile sensors offer a potential improvement, being durable, low cost, accurate and high bandwidth, but they are relatively undeveloped because of the complexities involved in design and calibration. This paper presents a general design methodology for magnetic field-based three-axis soft tactile sensors, enabling researchers to easily develop specific tactile sensors for a variety of applications. All aspects (design, fabrication, calibration and evaluation) of the development of tri-axis soft tactile sensors are presented and discussed. A moving least square approach is used to decouple and convert the magnetic field signal to force output to eliminate non-linearity and cross-talk effects. A case study of a tactile sensor prototype, MagOne, was developed. This achieved a resolution of 1.42 mN in normal force measurement (0.71 mN in shear force), good output repeatability and has a maximum hysteresis error of 3.4%. These results outperform comparable sensors reported previously, highlighting the efficacy of our methodology for sensor design.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Fingertip tactile sensation via piezoelectric micromachined ultrasonic transducers with an amplified interface

Junji Sone

DOI: https://doi.org/10.1038/s41598-024-52630-2

IF: 4.6

2024-02-02

Scientific Reports

Abstract:Tactile devices are often used in the field of robotics; however, the development of compact high-resolution tactile devices remains challenging. In this study, we developed a haptic device for force presentation using a DC motor and a tactile sensation device to simultaneously present haptic and tactile stimuli. A microelectromechanical system was selected to maintain the compactness of the tactile device. Piezoelectric micromachined ultrasonic transducers are known for high-power stimulation, and we selected lanthanum-doped lead zirconate titanate as the high-power amplified actuator. A finger mount structure that transfers force for amplifying ultrasonic waves was considered to combine acoustic pressure and aeroacoustics by attaching silicone rubber. The device was fabricated, and the performance of the tactile sensations was evaluated. The developed device uses the novel concept of combining acoustic pressure and aeroacoustics, and its compactness renders it suitable for wearable systems.

multidisciplinary sciences
Ghostfinger: a novel platform for fully computational fingertip controllers

Staas de Jong

DOI: https://doi.org/10.5281/zenodo.1176292

2021-07-27

Abstract:We present Ghostfinger, a technology for highly dynamic up/down fingertip haptics and control. The overall user experience offered by the technology can be described as that of tangibly and audibly interacting with a small hologram. More specifically, Ghostfinger implements automatic visualization of the dynamic instantiation/parametrization of algorithmic primitives that together determine the current haptic conditions for fingertip action. Some aspects of this visualization are visuospatial: A floating see-through cursor provides real-time, to-scale display of the fingerpad transducer, as it is being moved by the user. Simultaneously, each haptic primitive instance is represented by a floating block shape, type-colored, variably transparent, and possibly overlapping with other such block shapes. Further aspects of visualization are symbolic: Each instance is also represented by a type symbol, lighting up within a grid if the instance is providing output to the user. We discuss the system's user interface, programming interface, and potential applications. This is done from a general perspective that articulates and emphasizes the uniquely enabling role of the principle of computation in the implementation of new forms of instrumental control of musical sound. Beyond the currently presented technology, this also reflects more broadly on the role of Digital Musical Instruments (DMIs) in NIME.

Human-Computer Interaction
Dynamic Shaping of Multi-Touch Stimuli by Programmable Acoustic Metamaterial

Thomas Daunizeau,Sinan Haliyo,David Gueorguiev,Vincent Hayward

2024-08-19

Abstract:Acoustic metamaterials are artificial structures, often lattice of resonators, with unusual properties. They can be engineered to stop wave propagation in specific frequency bands. Once manufactured, their dispersive qualities remain invariant in time and space, limiting their practical use. Actively tuned arrangements have received growing interest to address this issue. Here, we introduce a new class of active metamaterial made from dual-state unit cells, either vibration sources when powered or passive resonators when left disconnected. They possess self-tuning capabilities, enabling deep subwavelength band gaps to automatically match the carrier signal of powered cells, typically around 200Hz. Swift electronic commutations between both states establish the basis for real-time reconfiguration of waveguides and shaping of vibration patterns. A series of experiments highlight how these tailored acceleration fields can spatially encode information relevant to human touch. This novel metamaterial can readily be made using off-the-shelf smartphone vibration motors, paving the way for a widespread adoption of multi-touch tactile displays.

Applied Physics,Human-Computer Interaction
Open Tactile - An open, modular hardware system for controlling tactile displays

Andreas Tarnowsky,Jan Jamaszyk,Daniel Brandes,Franz-Erich Wolter

DOI: https://doi.org/10.48550/arXiv.1804.08895

IF: 6.4588

2018-04-24

Human-Computer Interaction

Abstract:Tactile displays have a wide potential field of applications, ranging from enhancing Virtual-Reality scenarios up to aiding telesurgery as well as in fundamental psychological and neurophysiological research. In this paper, we describe an open source hardware and software architecture that is designed to drive a variety of different tactile displays. For demonstration purposes, a tactile computer mouse featuring a simple tactile display, based on lateral piezoelectric (PZT) actuators, is presented. Even though we will focus on driving mechanical actuators in this paper, the system can be extended to different working principles. The suggested architecture is supplied with a custom, easy to use, software stack allowing a simple definition of tactile scenarios as well as user studies while being especially tailored to non-computer scientists. By releasing the OpenTactile system under MIT license we hope to ease the burden of controlling tactile displays as well as designing and reproducing the related experiments.
KiloHertz Bandwidth, Dual-Stage Haptic Device Lets You Touch Brownian Motion

Tianming Lu,Cecile Pacoret,David Heriban,Abdenbi Mohand-Ousaid,Stephane Regnier,Vincent Hayward

DOI: https://doi.org/10.1109/TOH.2016.2644613

Abstract:This paper describes a haptic interface that has a uniform response over the entire human tactile frequency range. Structural mechanics makes it very difficult to implement articulated mechanical systems that can transmit high frequency signals. Here, we separated the frequency range into two frequency bands. The lower band is within the first structural mode of the corresponding haptic device while the higher one can be transmitted accurately by a fast actuator operating from conservation of momentum, that is, without reaction forces to the ground. To couple the two systems, we adopted a channel separation approach akin to that employed in the design of acoustic reproduction systems. The two channels are recombined at the tip of the device to give a uniform frequency response from DC to one kHz. In terms of mechanical design, the high-frequency transducer was embedded inside the tip of the main stage so that during operation, the human operator has only to interact with a single finger interface. In order to exemplify the type of application that would benefit from this kind of interface, we applied it to the haptic exploration with microscopic scales objects which are known to behave with very fast dynamics. The novel haptic interface was bilaterally coupled with a micromanipulation platform to demonstrate its capabilities. Operators could feel interaction forces arising from contact as well as those resulting from Brownian motion and could manoeuvre a micro bead in the absence of vision.

A tactile closed-loop device for musical interaction

The cyclotactor: towards a tactile platform for musical interaction

Tangible Music Interfaces Using Passive Magnetic Tags

Developing the cyclotactor

A Magnet‐Driven Soft Bistable Actuator

Generating Clear Vibrotactile Cues with a Magnet Embedded in a Soft Finger Sheath

A Touch on Musical Innovation: Exploring Wearables and Their Impact on New Interfaces for Musical Expression

A Soft Tactile Sensor Based on Magnetics and Hybrid Flexible-Rigid Electronics

Tactile Melodies: A Desk-Mounted Haptics for Perceiving Musical Experiences

A Mechatronic Platform for Studies of Tactile Sensory System on Prosthesis Hand

A Virtual 2D Tactile Array for Soft Actuators Using Acoustic Sensing

Magnet Haptic: Using Magnets and Conductive Ink to Self-fabricate Interactive VR Devices

Musinger: Communication of Music over a Distance with Wearable Haptic Display and Touch Sensitive Surface

Using Tangible Interaction to Design Musicking Artifacts for Non-musicians

Handheld Haptic Device with Coupled Bidirectional Input

Design Methodology for Magnetic Field-Based Soft Tri-Axis Tactile Sensors

Fingertip tactile sensation via piezoelectric micromachined ultrasonic transducers with an amplified interface

Ghostfinger: a novel platform for fully computational fingertip controllers

Dynamic Shaping of Multi-Touch Stimuli by Programmable Acoustic Metamaterial

Open Tactile - An open, modular hardware system for controlling tactile displays

KiloHertz Bandwidth, Dual-Stage Haptic Device Lets You Touch Brownian Motion