Abstract:Multi-modal bio-sensing has recently been used as effective research tools in affective computing, autism, clinical disorders, and virtual reality among other areas. However, none of the existing bio-sensing systems support multi-modality in a wearable manner outside well-controlled laboratory environments with research-grade measurements. This paper attempts to bridge this gap by developing a wearable multi-modal bio-sensing system capable of collecting, synchronizing, recording, and transmitting data from multiple bio-sensors: PPG, EEG, eye-gaze headset, body motion capture, GSR, etc., while also providing task modulation features including visual-stimulus tagging. This study describes the development and integration of various components of our system. We evaluate the developed sensors by comparing their measurements to those obtained by a standard research-grade bio-sensors. We first evaluate different sensor modalities of our headset, namely, earlobe-based PPG module with motion-noise canceling for ECG during heart-beat calculation. We also compare the steady-state visually evoked potentials measured by our shielded dry EEG sensors with the potentials obtained by commercially available dry EEG sensors. We also investigate the effect of head movements on the accuracy and precision of our wearable eye-gaze system. Furthermore, we carry out two practical tasks to demonstrate the applications of using multiple sensor modalities for exploring previously unanswerable questions in bio-sensing. Specifically, utilizing bio-sensing, we show which strategy works best for playing "Where is Waldo?" visual-search game, changes in EEG corresponding to true vs. false target fixations in this game, and predicting the loss/draw/win states through bio-sensing modalities while learning their limitations in a "Rock-Paper-Scissors" game.

Micro-Controllable, Multi-Functional Interface Module for Digital MP: A Wearable Computer Security Application

A Wearable System for Olfactory Electrophysiological Recording and Animal Motion Control

Development of an Eog (Electro-Oculography) Based Human-Computer Interface

A Portable Wireless Eye Movement-Controlled Human-Computer Interface for the Disabled

Wearable Multimedia System

A wearable computing system for wireless communication and gesture based human computer interface

ElectroDermis - Fully Untethered, Stretchable, and Highly-Customizable Electronic Bandages.

A Single-Channel and Non-Invasive Wearable Brain-Computer Interface for Industry and Healthcare

Portable Head-Mounted System for Mobile Forearm Tracking

Wearable Sensors‐Enabled Human–Machine Interaction Systems: From Design to Application

A Wearable Multi-Modal Bio-Sensing System Towards Real-World Applications

Fully Photonic Integrated Wearable Optical Interrogator

A Lightweight and Affordable Wearable Haptic Controller for Robot-Assisted Microsurgery

Design and Implementation of Multimedia Control System Based on Bluetooth and Electrooculogram (EOG)

A Multipurpose Human–Machine Interface via 3D-Printed Pressure-Based Force Myography

Toward the Next Generation Human-Machine Interaction: Headworn Wearable Devices

An historical account of the 'WearComp' and 'WearCam' inventions developed for applications in 'personal imaging'

Ordinateur porté support de réalité augmentée pour des activités de maintenance et de dépannage

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

A Wearable Ultrasound Interface for Prosthetic Hand Control

Minimalist and multi-functional human machine interface (HMI) using a flexible wearable triboelectric patch