Abstract:Objective Existing artificial vision devices can be divided into two types:implanted devices and extracorporeal devices,both of which have some disadvantages.The former requires surgical implantation,which may lead to irreversible trauma,while the latter has some defects such as relatively simple instructions,limited application scenarios and relying too much on the judgment of artificial intelligence(AI)to provide enough security.Here we propose a system that has voice interaction and can convert surrounding environment information into tactile commands on head and neck.Compared with existing extracorporeal devices,our device can provide a larger capacity of information and has advantages such as lower cost,lower risk,suitable for a variety of life and work scenarios.Methods With the latest remote wireless communication and chip technologies,microelectronic devices,cameras and sensors worn by the user,as well as the huge database and computing power in the cloud,the backend staff can get a full insight into the scenario,environmental parameters and status of the user remotely(for example,across the city)in real time.In the meanwhile,by comparing the cloud database and in-memory database and with the help of AI-assisted recognition and manual analysis,they can quickly develop the most reasonable action plan and send instructions to the user.In addition,the backend staff can provide humanistic care and emotional sustenance through voice dialogs.Results This study originally proposes the concept of"remote virtual companion"and demonstrates the related hardware and software as well as test results.The system can not only achieve basic guide functions,for example,helping a person with visual impairment to shop in supermarkets,find seats at cafes,walk on the streets,construct complex puzzles,and play cards,but also can meet the demand for fast-paced daily tasks such as cycling.Conclusion Experimental results show that this"remote virtual companion"is applicable for various scenarios and demands.It can help blind people with their travels,shopping and entertainment,or accompany the elderlies with their trips,wilderness explorations,and travels.

SeeSay: An Assistive Device for the Visually Impaired Using Retrieval Augmented Generation

An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

BlueView: a perception assistant system for the visually impaired.

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Hybrid auditory feedback: a new method for mobility assistance of the visually impaired.

Right mix of speech and non-speech: hybrid auditory feedback in mobility assistance of the visually impaired

TEXT2TASTE: A Versatile Egocentric Vision System for Intelligent Reading Assistance Using Large Language Model

A Multimodal Assistive System for Helping Visually Impaired in Social Interactions

Emerging Practices for Large Multimodal Model (LMM) Assistance for People with Visual Impairments: Implications for Design

EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments

Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models

Voice Assistant for Blind Person

Remote Virtual Companion Via Tactile Codes and Voices for the People with Visual Impairment

VIAssist: Adapting Multi-modal Large Language Models for Users with Visual Impairments

Conversations for Vision: Remote Sighted Assistants Helping People with Visual Impairments

A wearable assistive system for the visually impaired using object detection, distance measurement and tactile presentation

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant

A Wearable Visually Impaired Assistive System Based on Semantic Vision SLAM for Grasping Operation

GenAssist: Making Image Generation Accessible

Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines

By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting