Abstract:Due to the limitations in the perception of graphical information by blind people and the need to substitute the sense of sight with other senses, the correct use of multimedia in the presentation of graphics is particularly important. The aim of the authors was to correctly present visual information in the tactile and audio form and to provide contextually selected information to reduce the existing cognitive barriers. In this paper, the authors decided to research the method of exploring a tactile picture by a blind person and providing contextual and semantic information about the touched image elements using the developed tool for multimodal presentation of interactive audio-tactile graphics supporting the perception of blind people. The use of multimedia should improve the perception of the conveyed content. Therefore, the effectiveness of interpreting the information contained in the tactile image is verified by analyzing the tactile, audio, and contextual perceptions during the experiments. The following issues were considered in detail from the point of view of the blind and visually impaired: the recognition of shapes of image elements depending on their size and properties, the optimal width of elements displayed on the tablet screen, the time intervals between taps on an image element, the acceptable length of graphic element audio description. The results from this study suggest the need to select the image presentation parameters in various perception channels to the user's needs. The findings of our research on tactile shape perception of geometric figures indicate potential problems in recognizing the shapes of figures in the case of wrong preparation of tactile pictures. In the case of a small difference in the proportions of the length of the sides of the figure or a slight difference in the measures of the angles of the figure sides, we have noticed that most blind people fail to recognize its shape. Considerable progress in improving such perception can be achieved by increasing the proportions in the lengths of the sides and the measures of the angles between the sides of the figure. Further steps concern introducing an alternative audio description of the properties of the figure so as to improve the interpretability of the figure shape. For most users, the width of a virtual line in a digital image displayed on a tablet should be around 5 mm for a corresponding 1 mm tactile line. The configuration of the time intervals defining the user's gestures (2-taps, 3-taps) should be about 340 ms. Applying audio descriptions to tactile picture elements improves the understanding and interpretation of the information presented on it. Most participants in the test group accepted 5 to 10 seconds of voice prompts. Longer messages were incomprehensible, and details were hard to remember. In the proposed solution, the audio description can be divided into 2 or 3 different descriptions available to the user after tapping 2 or 3 times on an image element. It allows the user to decide on the amount of contextual information needed about the touched tactile picture element. The research attempted to show the typical values of various image presentation parameters and their ranges of values and indicated effective methods of their selection for a specific system user.

Audio-visual training and feedback to learn touch-based gestures

Right mix of speech and non-speech: hybrid auditory feedback in mobility assistance of the visually impaired

Hybrid auditory feedback: a new method for mobility assistance of the visually impaired.

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

The Influence of User-Tracking Feedback Format on Gestural Interaction's User Experience:A Kinect-Based Usability Study

Research On Visual Feedback Based On Natural Gesture

Interactive Design With Gesture and Voice Recognition in Virtual Teaching Environments

Use of Sound to Provide Occluded Visual Information in Touch Gestural Interface

User Training with Error Augmentation for Electromyogram-based Gesture Classification

Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind People

An investigation into the effectiveness of using acoustic touch to assist people who are blind

Teleguidance-based remote navigation assistance for visually impaired and blind people-usability and user experience

Perceiving Graphical and Pictorial Information via Hearing and Touch

SONOICE! a Sonar-Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers

Enriching Verbal Feedback from Usability Testing: Automatic Linking of Thinking-Aloud Recordings and Stimulus using Eye Tracking and Mouse Data

User Training With Error Augmentation for sEMG-Based Gesture Classification

Gesture Recognition Algorithm for Visually Blind Touch Interaction Optimization Using Crow Search Method

Helping Blind People Grasp: Evaluating a Tactile Bracelet for Remotely Guiding Grasping Movements

See or Hear? Exploring the Effect of Visual and Audio Hints and Gaze-assisted Task Feedback for Visual Search Tasks in Augmented Reality

Haptic Feedback to Assist Blind People in Indoor Environment Using Vibration Patterns

A Multimodal Assistive System for Helping Visually Impaired in Social Interactions