Function Features of the Human-Machine Interaction in the Computer Pan-Vision System

MO Jinqiu,ZHOU Xiaojun

DOI: https://doi.org/10.3321/j.issn:1004-132x.2000.05.018

2000-01-01

Abstract:Based on the fundamental ideas of computer pan-vision,this paper gives the function features of the human-machine interaction under the computer pan -vision system,the visual fusing, the plentiful visual way, and also the initiative and gradually advance.

What problem does this paper attempt to address?

Information Fusion Methods in Computer Pan-vision System

Zhou Xiaojun,Xu Zhinong,Cheng Yaodong

DOI: https://doi.org/10.3772/j.issn.1006-6748.2002.02.020

2002-01-01

Abstract:Aiming at concrete tasks of information fusion in computer pan-vision (CPV) system, information fusion methods are studied thoroughly. Some research progresses are presented. Recognizing of vision testing object is realized by fusing vision information and non-vision auxiliary information, which contain recognition of material defects, intelligent robot's autonomous recognition for parts and computer to defect image understanding and recognition automatically.
Human-Computer Interaction System Based on Binocular Stereo Vision

Liao Guangjun,Hu Yueming,Li Zhifu,Wu Xuedong

2012-01-01

Abstract:Binocular stereo vision is an important aspect in computer vision field. In this paper, it is made a systematic research, which is the system of human-computer interaction based on bincular stereo vision. In order to improve the anti-interference performance the hardware system worked in the condition of near infrared ; to inhance the real time operation , the level of difficulty with stereo matching algorithm is simplified through special hardware design; based on the ICP algorithm for tracking tools, the flexibility and convenience of system are increasingly. The application in surgical navigation system is discussed. In the prototype system, how to unified virtual coordinates and real coordinates has been presented; the problem on how to tracking patient undergoing in real-time is solved. The result has higher precision and practicability, which provide a good platform for virtual reality.
A Present and Development of Catadioptric Panoramic Stereo Imaging Technique

陈炫屹,白剑

DOI: https://doi.org/10.3969/j.issn.1005-5630.2009.06.018

2009-01-01

Abstract:Catadioptric panoramic stereo imaging technique plays important roles in the applications of machine vision, virtual reality, etc. Catadioptric panoramic stereo imaging technique is based on the catadioptric panoramic imaging technology for large field of view imaging,and the principle of binocular vision to obtain stereo information. In this paper, the present and development of catadioptric panoramic stereo imaging technique is expounded systematically, and the advantages and disadvantages of some important approaches are discussed. Finally, the existing problems and future development in catadioptric panoramic stereo imaging technique are discussed.
A Visual Human-Computer Interaction System Based on Hybrid Visual Model

Ziyi Ke,Ziqiang Chen,Huanlei Wang,Liang Yin

DOI: https://doi.org/10.1155/2022/9562104

IF: 1.968

2022-07-01

Security and Communication Networks

Abstract:The traditional human-computer interaction is mainly through the mouse, keyboard, remote control, and other peripheral equipment electromagnetic signal transmission. This paper aims to build a visual human-computer interaction system through a series of deep learning and machine vision models, so that people can achieve complete human-computer interaction only through the camera and screen. The established visual human-computer interaction system mainly includes the function modes of three basic peripherals in human-computer interaction: keyboard, mouse (X-Y position indicator), and remote control. The convex hull method was used to switch between these three modes. After issuing the mode command, Gaussian mixture was used to quickly identify the moving human body to narrow the scope of our image processing. Subsequently, finger detection in human body was realized based on the Faster-RCNN-ResNET50-FPN model structure, and realized the function of moving mouse and keyboard through the relationship between different fingers. At the same time, the recognition of human body posture was done by using MediaPipe BlazePose, and the action classification models were established through the Angle between body movements so as to realize the control function of remote control. In order to ensure the real-time performance of the interactive system, according to the characteristics of different data processing processes, CPU and GPU computing power resources are used to cross-process images to ensure the real-time performance. The recognition accuracy of the human-computer interaction system is above 0.9 for the key feature points of human body, and above 0.87 for the recognition accuracy of four kinds of command actions. It is hoped that vision-based human-computer interaction will become a widely used interaction mode in the future.

computer science, information systems,telecommunications
Human-Machine Plan Conflict and Conflict Resolution in a Visual Search Task

Yunxian Pan,Jie Xu

DOI: https://doi.org/10.1016/j.ijhcs.2024.103377

2025-01-01

Abstract:With rapid technological development, humans are more likely to cooperatively work with intelligence systems in everyday life and work. Similar to interpersonal teamwork, the effectiveness of human-machine teams is affected by conflicts. Some human-machine conflict scenarios occur when neither the human nor the system was at fault, for example, when the human and the system formulated different but equally effective plans to achieve the same goal. In this study, we conducted two experiments to explore the effects of human-machine plan conflict and the different conflict resolution approaches (human adapting to the system, system adapting to the human, and transparency design) in a computer-aided visual search task. The results of the first experiment showed that when conflicts occurred, the participants reported higher mental load during the task, performed worse, and provided lower subjective evaluations towards the aid. The second experiment showed that all three conflict resolution approaches were effective in maintaining task performance, however, only the transparency design and the human adapting to the system approaches were effective in reducing mental load and improving subjective evaluations. The results highlighted the need to design appropriate human-machine conflict resolution strategies to optimize system performance and user experience.
Motion Control System Design of a Humanoid Robot Based on Stereo Vision

Qinjun Du,Chao Sun,Xingguo Huang

DOI: https://doi.org/10.4028/www.scientific.net/amm.55-57.877

2011-01-01

Applied Mechanics and Materials

Abstract:Vision is an important means of the humanoid robot to get external environmental information; vision system is an important part of humanoid robot. The system of a humanoid robot with the functions of visual perception and object manipulation is very complex because the body of the humanoid robot possesses many joint units and sensors. Two computers linked by Memolink communication unit is adopted to meet the needs of real time motion control and visual information processing tasks. The motion control system included coordination control computer, the distributed DSP joint controllers, DC motor drivers and sensors. Linux and real-time RT-Linux OS are used as the operating system to achieve the real-time control capability.
A Multi-modal Virtual-Real Fusion System for Multi-task Human-Computer Interaction.

Xinlei Zhang,Jiahui Yu,Yuxiang Sun,Min Li,Yang Song,Xianzhong Zhou

DOI: https://doi.org/10.1109/icnsc55942.2022.10004097

2022-01-01

Abstract:Due to the complexity of the task and the diversification of the scene, the traditional interactive control method can no longer meet the requirements of users. To solve this problem, a multi-modal virtual-real fusion system for multi-task human-computer interaction is proposed in this paper, which integrates eye movement, gesture and voice. Frist, aiming at the phenomenon of multi-task and multi-modal, a task-modal matching model is established. Then, the task-modal matching model is abstracted into a multi-objective optimization problem, and a method for solving this problem is designed and a matching scheme is successfully obtained. Meanwhile, the construction of the system is completed for the virtual-real fusion environment, and the control of unmanned car and virtual car is realized. The system can carry out multi-modal interaction and complete multiple tasks in real scene, virtual scene, parallel system and virtual-real fusion scene. Finally, the experiment proves the stability and reliability of the system.
An Adaptive Vision System Toward Implicit Human Computer Interaction

Peng Dai,Linmi Tao,Xiang Zhang,Ligeng Dong,Guangyou Xu

DOI: https://doi.org/10.1007/978-3-540-73281-5_87

2007-01-01

Abstract:In implicit human computer interaction, computers are required to understand users' actions and intentions so as to provide proactive services. Visual, processing has to detect and understand human actions and then transform them as the implicit input. In this paper an adaptive vision system is presented to solve visual processing tasks in dynamic meeting context. Visual modules and dynamic context analysis tasks are organized in a bidirectional scheme. Firstly human objects are detected and tracked to generate global features. Secondly current meeting scenario is inferred based on these global features, and in some specific scenarios face and hand blob level visual processing tasks are fulfilled to extract visual information for the analysis of individual and interactive events, which can further be adopted as implicit input to the computer system. The experiments in our smart meeting room demonstrate the effectiveness of the proposed framework.
Virtual-Physical Interaction System

Xin Min,Shouqian Sun,Wenqiao Zhang,Chao Li

DOI: https://doi.org/10.1109/ICVRV.2018.00056

2018-01-01

Abstract:We present a novel human-computer interaction (HCI) system called Virtual-Physical Interaction System (VPIS): On the one hand, offering a realistic product experience process in a Virtual Reality (VR) environment. On the other hand, the corresponding process is controlled by manipulating a tangibly product model in a physical world. Thus, the VPIS with a Virtual-Physical combination enhances user experience in HCI field, which can apply in several application scenarios (Aided Design). However, the implementation of this system is a challenging task that subsumes several fundamental problems as sub-tasks: target detection and tracking, gesture detection and recognition. In order to achieve the expected goal of VPIS, the system is based on the physical hardware (HoloLens, Leap Motion and 3D printer) and the existing computer vision (CV) algorithms.
Human-computer interaction based on machine vision of a smart assembly workbench

Shiqing Wu,Zhonghou Wang,Bin Shen,Jia-Hai Wang,Li Dongdong

DOI: https://doi.org/10.1108/aa-10-2018-0170

IF: 2.373

2020-01-25

Assembly Automation

Abstract:Purpose The purpose of this study is to achieve multi-variety and small-batch assembly through direct cooperation between equipment and people and to improve assembly efficiency as well as flexibility. Design/methodology/approach Firstly, the concept of the human–computer interaction is designed. Secondly, the machine vision technology is studied theoretically. Skin color filter based on hue, saturation and value color model is put forward to screen out images that meet the skin color characteristics of the worker, and a multi-Gaussian weighted model is built to separate moving objects from its background. Both of them are combined to obtain the final images of the target objects. Then, the key technology is applied to the smart assembly workbench. Finally, experiments are conducted to evaluate the role of the human–computer interaction features in improving productivity for the smart assembly workbench. Findings The result shows that multi-variety and small-batch considerable increases assembly time and the developed human–computer interaction features, including prompting and introduction, effectively decrease assembly time. Originality/value This study proves that the machine vision technology studied in this paper can effectively eliminate the interferences of the environment to obtain the target image. By adopting the human–computer interaction features, including prompting and introduction, the efficiency of manual operation is improved greatly, especially for multi-variety and small-batch assembly.

automation & control systems,engineering, manufacturing
Improvement on Panoramic Camera Control in 3D Management Information System

MA Jing-heng,ZHUANG Hui,XIANG Min,LI Yu

DOI: https://doi.org/10.3969/j.issn.1672-7843.2010.06.017

2010-01-01

Abstract:As a media in interaction between human and computer,camera directly determines the visual effects for the scene in three-dimensional management information systems.Based on the analysis of two aspects of position and orientation on how to select the appropriate cameras,two sorts of methods on camera control are proposed.By use of Virtools for programming,and testing,it shows that the use of the improved camera control operator can avoid complicated keyboard manipulations on viewing terrain,reduce the performance of the hardware configuration requirements,enhance users' visual effects for scene,and improve management efficiency.
A Software Approach Of Human-Machine Interface Ergonomics Evaluation

Hongjun Xue,Ye Yuan,Jiayu Chen,Xiaoyan Zhang

DOI: https://doi.org/10.1007/978-981-13-8779-1_70

2020-01-01

Abstract:Human-machine interface ergonomics design and evaluation can not only affect the efficiency of interaction, but also affect the operation safety. Therefore, it is necessary to develop an effective human-machine ergonomics evaluation method to evaluate the human-machine ergonomics design. This paper proposed a human-machine interface evaluation criterion based on visual perception intensity and a contrast criterion for image processing, and built an evaluation software. The software built was based on Microsoft Visual Studio 2010. Besides, the interface of the software was written in C++ programming language. The software included three functional modules: calculating the interface visual perception index, calculating the interface contrast ratio, and the interface ergonomics evaluation result. The simulation results suggested that the software built can be used to evaluate the human-machine interface and can effectively avoid the potential subjective influence of the experimenter in the traditional evaluation.
Human–computer interaction based visual feedback system for augmentative and alternative communication

Yubin Liu,C. B. Sivaparthipan,Achyut Shankar,Yubin Liu,C. B. Sivaparthipan,Achyut Shankar

DOI: https://doi.org/10.1007/s10772-021-09901-4

2021-10-18

International Journal of Speech Technology

Abstract:The knowledgeable, human–machine interaction sight system has the benefits of low interference, lower permeability, and no interface attachment. The smart vision system has been critical in human–computer interaction to grow and advance technologies and research. The Human–Computer Interaction based Visual Feedback System (HCIVFS) is very quickly relative to the conventional collaborative mode. Such challenges may also affect the smart machine's view and the general use of sensation communication. The fundamental premise of the computer's sight communication architecture requires practical stability. This article explores the quality of the intellectual computer's enabling communication. The Rule of Fitts has also been included in this paper for three points-to-clicks applications. The proposed algorithm's reliability is analyzed in operations, visualization, and computer vision algorithms. There is a fair recommendation for an immersive configuration of the input method for intellectual sight by computer.
Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

Yameng Yang,Yu Zhang

DOI: https://doi.org/10.1080/10447318.2024.2307689

IF: 4.92

2024-02-15

International Journal of Human-Computer Interaction

Abstract:A flexible production model in which humans and robots collaborate has become an urgent need for industrial production. Therefore, research on human-machine integration systems has received more and more attention in recent years. Human-computer interaction is widely used in industrial production and entertainment life, such as industrial control, VR games, etc. In this article, we designed and implemented a human-machine fusion system that meets different interactive tasks. Human movement estimation and robot interactive control are the key technologies. First, we propose a human posture solution method based on information fusion of multiple cameras and inertial measurement units. This method constructs an optimization problem, fuses the 2D joint detection information under multiple cameras and the measurement information of the worn inertial measurement unit, and optimizes the estimation of human kinematic posture, which improves the problems of incomplete posture information and sensitivity to noise under a single sensor. Therefore, this method improves the accuracy of pose estimation. Secondly, combining the kinematic characteristics of the robot and the characteristics of human-computer interaction, we designed a robot control strategy based on target point tracking and model predictive control, so that the robot can adapt to dynamic environments and different interaction needs by adjusting control parameters. At the same time, our design ensures the safety of robots and operators. Finally, we conducted human-computer interaction experiments such as action following, item delivery, and active obstacle avoidance. The experimental results show the effectiveness and reliability of the designed robot interaction system in a human-machine fusion environment.

computer science, cybernetics,ergonomics
A Solution of Human-Computer Remote Interaction with Tactile Feedback

Yanan Chen,Shouxiang Ni,Mingkai Chen,Jianxin Chen

DOI: https://doi.org/10.1109/IWCMC51323.2021.9498764

2021-06-28

Abstract:In human-computer interaction, we need to use human's multiple sensory channels and motion channels to improve its naturalness and efficiency. These channels include voice, handwriting, posture, vision, expression, etc. Visual-based human-computer interaction systems are often mentioned in the literature. The system uses a camera to monitor and track the operator's posture, and sets corresponding commands to control the movement of the robot. On top of that, we add tactile feedback to allow the robot to accomplish more tasks. In the vision system, first we obtain color images and depth images through a depth camera. Secondly, we estimate the human posture through OpenPose. Finally, we map the motion of the human arm to the motion of the robotic arm. In the tactile system, first we control the movement of the magic hand through the exoskeleton. Secondly, we obtain the force values of the fingers through the magic hand. Finally, we provide force feedback to each finger through the exoskeleton.

Engineering,Computer Science
Summary of the Research on the Interaction of Tightly Coupled Physical Human-machine System

FANG Bin,SUN Baixin,CHENG Guang,DAI Quanmin,LIU Weifeng,SUN Fuchun

DOI: https://doi.org/10.3901/jme.2021.17.010

2021-01-01

Journal of Mechanical Engineering

Abstract:： Tightly coupled physical man-machine systems are used to enhance and reconstruct human motion functions, and have been widely used in fields such as helping the elderly and the disabled, rehabilitation medicine, and national defense and military. The paper surveys on the interaction of tightly coupled physical man-machine systems. Through functional division, the system is divided into three aspects: functional replacement, functional rehabilitation and functional enhancement, and the research status of each type of system is introduced in detail. Then, the status quo of human-computer interaction interface is explained from two aspects of implanted sensing and external sensing. Then the perceptual intention recognition algorithm of tightly coupled human-computer interaction are analyzed. Finally, the future development is prospected and summarized.
Structural design of anthropomorphic robot vision system

Junyan Zhang

DOI: https://doi.org/10.1088/1742-6596/1423/1/012055

2019-12-01

Journal of Physics: Conference Series

Abstract:Abstract Anthropomorphic robot is one of the latest research directions in the field of intelligent robot research. It is an intelligent robot with similar shape and humanity. It has mobile function, operation function, sensing function, memory and autonomy, and can realize human-computer interaction. Robot technology has always been a hotspot of technology that people pay attention to. Because of the diverse development of robots, machine vision systems are gradually being established. The machine vision system realizes the functions of collecting images, analyzing images, acquiring information, processing images and others by simulating the human eye. It is a comprehensive technology integrating automation and computer. The article deeply studies the hardware composition and image processing technology of machine vision system and gives an example study on the application of machine vision in various fields. It also analyzes the current development status of machine vision system in China and looks forward to the bright future of machine vision.
Research on multimodal human-computer interaction technology based on audiovisual fusion

Zhinan Jiao

DOI: https://doi.org/10.1109/ICSP54964.2022.9778516

2022-04-15

Abstract:With the improvement of information technology, service robots are becoming more and more deeply involved in our work and life, and provide a wider variety of services. How to make robots communicate with humans more intelligently and understand and accomplish people's task needs has become a current research hotspot. In this paper, we propose a multimodal human-robot interaction technique with D-S evidence theory for fusion of audiovisual information, and a multimodal interaction system with speech and gesture interaction to improve the performance of human-robot interaction.

Engineering,Computer Science
Full-Field of View Display Combined with Motion Interaction Device

Dong Li,Jinghui Xie,Dongdong Weng,Yan Mo

DOI: https://doi.org/10.1109/vr.2013.6549375

2013-01-01

Abstract:Majority of present applications use buttons or touch panels as their human-computer interface. The limitation of these methods is that users can not interact with the equipments in a natural way, since people should operate the equipments according to certain rules which are arranged beforehand. This paper proposes a novel interaction system which contains a low cost full-field of view device to make sure users can see the display content from all directions. In addition, a tracking device is set up over users head to identify the user's natural gesture. A target following photography equipment is also combined with the system to provide on-line video chat and remote monitoring functions. Our system can be considered as a new form of human-computer interface and the advantages and potential are discussed in this paper. Finally, a demonstration and experimental results are given to evaluate the validity of the system.
Introducing the Knowledge of Human Vision into the Lecture of Computer Vision

LIU Ya-dong,LI Ming,ZHOU Zong-tan,HU De-wen

DOI: https://doi.org/10.3969/j.issn.1008-0686.2010.05.004

2010-01-01

Abstract:This paper discuss the knowledge of human vision into the lecture of Computer Vision course.The discussion is organized in three sections,namely sense,apperception and cognitions.The concrete joint points are also involved.By introducing the comparison between computer vision and human vision,we hope that better understanding involving the content,deficiency and possible improvement of computer vision techniques such as edge detection,region analysis,color recognition and stereopsis can be achieved.We also hope that students can achieve a more in-depth think in essence of computer vision.

Function Features of the Human-Machine Interaction in the Computer Pan-Vision System

Information Fusion Methods in Computer Pan-vision System

Human-Computer Interaction System Based on Binocular Stereo Vision

A Present and Development of Catadioptric Panoramic Stereo Imaging Technique

A Visual Human-Computer Interaction System Based on Hybrid Visual Model

Human-Machine Plan Conflict and Conflict Resolution in a Visual Search Task

Motion Control System Design of a Humanoid Robot Based on Stereo Vision

A Multi-modal Virtual-Real Fusion System for Multi-task Human-Computer Interaction.

An Adaptive Vision System Toward Implicit Human Computer Interaction

Virtual-Physical Interaction System

Human-computer interaction based on machine vision of a smart assembly workbench

Improvement on Panoramic Camera Control in 3D Management Information System

A Software Approach Of Human-Machine Interface Ergonomics Evaluation

Human–computer interaction based visual feedback system for augmentative and alternative communication

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

A Solution of Human-Computer Remote Interaction with Tactile Feedback

Summary of the Research on the Interaction of Tightly Coupled Physical Human-machine System

Structural design of anthropomorphic robot vision system

Research on multimodal human-computer interaction technology based on audiovisual fusion

Full-Field of View Display Combined with Motion Interaction Device

Introducing the Knowledge of Human Vision into the Lecture of Computer Vision