Machine Vision

D. Hughes

DOI: https://doi.org/10.12968/s1478-2774(23)50333-1

2023-03-01

Abstract:AI co-pilots could soon be improving safety in aircraft by providing collision avoidance and navigation prompts

Engineering,Computer Science

What problem does this paper attempt to address?

Traffic Collisions Early Warning Aided by Small Unmanned Aerial Vehicle Companion

Hao Luo,Shu-Chuan Chu,Xiaofeng Wu,Zhenfei Wang,Fangqian Xu

DOI: https://doi.org/10.1007/s11235-015-0131-5

2016-01-01

Telecommunication Systems

Abstract:Most traffic surveillance systems are based on videos which captured by fixed cameras on bridges, intersections, etc. However, many traffic collisions may occur in many places without such surveillance systems, e.g., in rural highway. Researchers have developed a set of techniques to improve safety on these places, while it is still not enough to reduce collision risk. Based on a novel concept, this paper proposes a traffic collisions early warning scheme aided by small unmanned aerial vehicle (UAV) companion. Basically, it is a vision-based driver assistance system, and the difference in comparison with the available schemes lies in the camera is flying along with the host vehicle. In particular, the system's framework and the vision-based vehicle collision detection algorithm are proposed. The small UAV works in two switchable modes, i.e., high speed flight or low speed motion. The high speed flight corresponds to the host vehicle moving in highway, while the low speed motion includes hover, vertical takeoff and landing. In addition, as the on-line machine learning is applied, the detection procedure can be implemented in real-time, which is critical in practical applications. Extensive experimental results and examples demonstrate the effectiveness of the proposed method, and its real-time performance outperforms typical tracking methods such as that based on Gaussian mixture model. Moreover, this scheme can be easily extended for some other similar application scenarios.
Enhancing Situational Awareness of Helicopter Pilots in Unmanned Aerial Vehicle-Congested Environments Using an Airborne Visual Artificial Intelligence Approach

John Mugabe,Mariusz Wisniewski,Adolfo Perrusquía,Weisi Guo

DOI: https://doi.org/10.3390/s24237762

IF: 3.9

2024-12-05

Sensors

Abstract:The use of drones or Unmanned Aerial Vehicles (UAVs) and other flying vehicles has increased exponentially in the last decade. These devices pose a serious threat to helicopter pilots who constantly seek to maintain situational awareness while flying to avoid objects that might lead to a collision. In this paper, an Airborne Visual Artificial Intelligence System is proposed that seeks to improve helicopter pilots' situational awareness (SA) under UAV-congested environments. Specifically, the system is capable of detecting UAVs, estimating their distance, predicting the probability of collision, and sending an alert to the pilot accordingly. To this end, we aim to combine the strengths of both spatial and temporal deep learning models and classic computer stereo vision to (1) estimate the depth of UAVs, (2) predict potential collisions with other UAVs in the sky, and (3) provide alerts for the pilot with regards to the drone that is likely to collide. The feasibility of integrating artificial intelligence into a comprehensive SA system is herein illustrated and can potentially contribute to the future of autonomous aircraft applications.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
The Role of Machine Vision for Intelligent Vehicles

Benjamin Ranft,Christoph Stiller

DOI: https://doi.org/10.1109/tiv.2016.2551553

IF: 8.2

2016-03-01

IEEE Transactions on Intelligent Vehicles

Abstract:Humans assimilate information from the traffic environment mainly through visual perception. Obviously, the dominant information required to conduct a vehicle can be acquired with visual sensors. However, in contrast to most other sensor principles, video signals contain relevant information in a highly indirect manner and hence visual sensing requires sophisticated machine vision and image understanding techniques. This paper provides an overview on the state of research in the field of machine vision for intelligent vehicles. The functional spectrum addressed covers the range from advanced driver assistance systems to autonomous driving. The organization of the article adopts the typical order in image processing pipelines that successively condense the rich information and vast amount of data in video sequences. Data-intensive low-level early vision techniques first extract features that are later grouped and further processed to obtain information of direct relevance for vehicle guidance. Recognition and classification schemes allow to identify specific objects in a traffic scene. Recently, semantic labeling techniques using convolutional neural networks have achieved impressive results in this field. High-level decisions of intelligent vehicles are often influenced by map data. The emerging role of machine vision in the mapping and localization process is illustrated at the example of autonomous driving. Scene representation methods are discussed that organize the information from all sensors and data sources and thus build the interface between perception and planning. Recently, vision benchmarks have been tailored to various tasks in traffic scene perception that provide a metric for the rich diversity of machine vision methods. Finally, the paper addresses computing architectures suited to real-time implementation. Throughout the paper, numerous specific examples and real world experiments with prototype vehicles are presented.
Machine vision-based autonomous road hazard avoidance system for self-driving vehicles

Chengqun Qiu,Hao Tang,Yuchen Yang,Xinshan Wan,Xixi Xu,Shengqiang Lin,Ziheng Lin,Mingyu Meng,Changli Zha

DOI: https://doi.org/10.1038/s41598-024-62629-4

IF: 4.6

2024-05-30

Scientific Reports

Abstract:The resolution of traffic congestion and personal safety issues holds paramount importance for human's life. The ability of an autonomous driving system to navigate complex road conditions is crucial. Deep learning has greatly facilitated machine vision perception in autonomous driving. Aiming at the problem of small target detection in traditional YOLOv5s, this paper proposes an optimized target detection algorithm. The C3 module on the algorithm's backbone is upgraded to the CBAMC3 module, introducing a novel GELU activation function and EfficiCIoU loss function, which accelerate convergence on position loss l box , confidence loss l obj , and classification loss l cls , enhance image learning capabilities and address the issue of inaccurate detection of small targets by improving the algorithm. Testing with a vehicle-mounted camera on a predefined route effectively identifies road vehicles and analyzes depth position information. The avoidance model, combined with Pure Pursuit and MPC control algorithms, exhibits more stable variations in vehicle speed, front-wheel steering angle, lateral acceleration, etc., compared to the non-optimized version. The robustness of the driving system's visual avoidance functionality is enhanced, further ameliorating congestion issues and ensuring personal safety.

multidisciplinary sciences
Artificial Intelligence in Adaptive In-Cab Interfaces

DOI: https://doi.org/10.30987/2658-4026-2023-4-351-360

2023-11-17

Ergodesign

Abstract:The principles of developing adaptive in-cabin control interfaces for advanced aviation systems based on the methodology of integrated modular avionics (IMA), which change their functionality depending on the situation and flight conditions, are considered. Existing and promising solutions and technologies for making in-cabin interfaces are described, such as gesture recognition systems, voice commands and other methods of interaction with the pilot, including the use of induced virtual reality and artificial intelligence technologies. The problems of piloting in conditions of increased automation of aircraft control processes are shown. Proposals are considered to create even more efficient and secure interfaces in the near future, which will improve the aviation system performance, the aircraft control quality and reduce the likelihood of pilots’ operational errors.
Machine Vision for UAS Ground Operations

Matthew Coombes,William Eaton,Wen-Hua Chen

DOI: https://doi.org/10.1007/s10846-017-0542-5

2017-03-31

Abstract:This paper discusses the machine vision element of a system designed to allow Unmanned Aerial System (UAS) to perform automated taxiing around civil aerodromes, with only a monocular camera. The purpose of the computer vision system is to provide direct sensor data which can be used to validate vehicle position, in addition to detecting potential collision risks. In practice, untrained clustering is used to segment the visual feed before descriptors of each cluster (primarily colour and texture) are used to estimate the class. As the competency of each individual estimate can vary dependent on multiple factors (number of pixels, lighting conditions and even surface type). A Bayesian network is used to perform probabilistic data fusion, in order to improve the classification results. This result is shown to perform accurate image segmentation in real-world conditions, providing information viable for localisation and obstacle detection.

robotics,computer science, artificial intelligence
Design of Autonomous Obstacle Avoidance System for Driverless Vehicle Based on Machine Vision

Shouqing Lu

DOI: https://doi.org/10.1145/3495018.3495519

2021-10-23

Abstract:With the continuous development of artificial intelligence technology, more and more automation technologies are combined with various machines. Automation technology has reached the driving field from the industrial revolution. Driverless technology has gradually become a major direction of the automobile market. The focus of driverless vehicles is the development of autonomous obstacle avoidance system. The perception system of driverless vehicles is responsible for sensing and interacting with the surrounding environment, usually including sensing information, data fusion, positioning and other functions. The reliability of machines is higher than that of people. In theory, automatic driving technology will be safer, and driverless vehicles have also proved this statistically. It can indeed make people's travel safer and reduce the probability of traffic accidents. Moreover, the autonomous obstacle avoidance system of driverless vehicle based on machine vision can greatly improve the overall traffic efficiency and greatly alleviate the traffic congestion. The experimental results show that the design of autonomous obstacle avoidance system for driverless vehicles based on machine vision reduces the road congestion rate by 8%. Machine vision enhances the ability of unmanned vehicles to successfully complete trajectory planning and following tasks when driving on any road, and its algorithm needs good real-time performance and stable reliability. For the limitations of the research on driverless mode, the method of innovative combination is adopted to update and iterate the autonomous obstacle avoidance of driverless vehicles, and the path planning for local navigation of various vehicles is analyzed, discussed and summarized, so as to enrich the research results of vehicle automatic obstacle avoidance system.
The Machine Vision Iceberg Explained: Advancing Dynamic Testing by Considering Holistic Environmental Relations

Hubert Padusinski,Christian Steinhauser,Thilo Braun,Lennart Ries,Eric Sax

2024-04-30

Abstract:Machine Vision (MV) is essential for solving driving automation. This paper examines potential shortcomings in current MV testing strategies for highly automated driving (HAD) systems. We argue for a more comprehensive understanding of the performance factors that must be considered during the MV evaluation process, noting that neglecting these factors can lead to significant risks. This is not only relevant to MV component testing, but also to integration testing. To illustrate this point, we draw an analogy to a ship navigating towards an iceberg to show potential hidden challenges in current MV testing strategies. The main contribution is a novel framework for black-box testing which observes environmental relations. This means it is designed to enhance MV assessments by considering the attributes and surroundings of relevant individual objects. The framework provides the identification of seven general concerns about the object recognition of MV, which are not addressed adequately in established test processes. To detect these deficits based on their performance factors, we propose the use of a taxonomy called "granularity orders" along with a graphical representation. This allows an identification of MV uncertainties across a range of driving scenarios. This approach aims to advance the precision, efficiency, and completeness of testing procedures for MV.

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Software Engineering,Image and Video Processing
Applications of Computer Vision in Autonomous Vehicles: Methods, Challenges and Future Directions

Xingshuai Dong,Massimiliano L. Cappuccio

2024-06-15

Abstract:Autonomous vehicle refers to a vehicle capable of perceiving its surrounding environment and driving with little or no human driver input. The perception system is a fundamental component which enables the autonomous vehicle to collect data and extract relevant information from the environment to drive safely. Benefit from the recent advances in computer vision, the perception task can be achieved by using sensors, such as camera, LiDAR, radar, and ultrasonic sensor. This paper reviews publications on computer vision and autonomous driving that are published during the last ten years. In particular, we first investigate the development of autonomous driving systems and summarize these systems that are developed by the major automotive manufacturers from different countries. Second, we investigate the sensors and benchmark data sets that are commonly utilized for autonomous driving. Then, a comprehensive overview of computer vision applications for autonomous driving such as depth estimation, object detection, lane detection, and traffic sign recognition are discussed. Additionally, we review public opinions and concerns on autonomous vehicles. Based on the discussion, we analyze the current technological challenges that autonomous vehicles meet with. Finally, we present our insights and point out some promising directions for future research. This paper will help the reader to understand autonomous vehicles from the perspectives of academia and industry.

Computer Vision and Pattern Recognition,Robotics
IVVI: Intelligent vehicle based on visual information

José María Armingol,Arturo de la Escalera,Cristina Hilario,Juan M. Collado,Juan Pablo Carrasco,Marco Javier Flores,José Manuel Pastor,Fo José Rodríguez

DOI: https://doi.org/10.1016/j.robot.2007.09.004

IF: 3.7

2007-12-01

Robotics and Autonomous Systems

Abstract:Human errors are the cause of most traffic accidents, with drivers’ inattention and wrong driving decisions being the two main sources. These errors can be reduced, but not completely eliminated. That is why Advanced Driver Assistance Systems (ADAS) can reduce the number, danger and severity of traffic accidents. Several ADAS, which nowadays are being researched for Intelligent vehicles, are based on Artificial Intelligence and Robotics technologies. In this article a research platform for the implementation of systems based on computer vision is presented, and different visual perception modules useful for some ADAS such as Line Keeping System, Adaptive Cruise Control, Pedestrian Protector, or Speed Supervisor, are described.

robotics,automation & control systems,computer science, artificial intelligence
The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond

Abigail Sellen,Eric Horvitz

DOI: https://doi.org/10.1145/3637865

IF: 22.7

2024-07-01

Communications of the ACM

Abstract:Building on cross-disciplinary insights to shape the future of human-AI interaction.

computer science, theory & methods, software engineering, hardware & architecture
Survey on Human-Vehicle Interactions and AI Collaboration for Optimal Decision-Making in Automated Driving

Abu Jafar Md Muzahid,Xiaopeng Zhao,Zhenbo Wang

2024-12-11

Abstract:The capabilities of automated vehicles are advancing rapidly, yet achieving full autonomy remains a significant challenge, requiring ongoing human cognition in decision-making processes. Incorporating human cognition into control algorithms has become increasingly important, as researchers work to develop strategies that minimize conflicts between human drivers and AI systems. Despite notable progress, many challenges persist, underscoring the need for further innovation and refinement in this field. This review covers recent progress in human-vehicle interaction (HVI) and AI collaboration for vehicle control. First, we start by looking at how HVI has evolved, pointing out key developments and identifying persistent problems. Second, we discuss the existing techniques, including methods for integrating human intuition and cognition into decision-making processes and developing systems that can mimic human behavior to enable optimal driving strategies and achieve safer and more efficient transportation. This review aims to contribute to the development of more effective and adaptive automated driving systems by enhancing human-AI collaboration.

Systems and Control
Vision-based intelligent vehicles: State of the art and perspectives

Massimo Bertozzi,Alberto Broggi,Alessandra Fascioli

DOI: https://doi.org/10.1016/s0921-8890(99)00125-6

IF: 3.7

2000-07-01

Robotics and Autonomous Systems

Abstract:Recently, a large emphasis has been devoted to Automatic Vehicle Guidance since the automation of driving tasks carries a large number of benefits, such as the optimization of the use of transport infrastructures, the improvement of mobility, the minimization of risks, travel time, and energy consumption.This paper surveys the most common approaches to the challenging task of Autonomous Road Following reviewing the most promising experimental solutions and prototypes developed worldwide using AI techniques to perceive the environmental situation by means of artificial vision.The most interesting results and trends in this field as well as the perspectives on the evolution of intelligent vehicles in the next decades are also sketched out.

robotics,automation & control systems,computer science, artificial intelligence
Machine visions: A corporate imaginary of artificial sight

Wei-Jie Hsiao,Samantha Shorey

DOI: https://doi.org/10.1177/14614448231176765

2023-06-24

Abstract:New Media &Society, Ahead of Print. Machine vision is one the most consequential applications of artificial intelligence (AI) in contemporary society. This article analyzes how companies that produce machine vision technology articulate what it means for machines to "see." Through a thematic analysis of more than 200 corporately produced documents, we examine the companies' product offerings and identify three discursive techniques that entwine basic explanations of emerging technology with the ideologies of AI producers: dismantling sight into technical action, expanding the parameters of sight, and seeing through data. These recurring corporate narratives organize perceptions of automation, educating outsiders how to value computational outcomes and support them through rearranging the real-world conditions of labor. We argue that the social power of machine vision is not only in how it detects objects, but also in how it arbitrates what work is visible in visions of the industry's future.

communication
Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles

Guangwei Yang,Christie Ridgeway,Andrew Miller,Abhijit Sarkar

DOI: https://doi.org/10.3390/s24082478

IF: 3.9

2024-04-13

Sensors

Abstract:Human factors are a primary cause of vehicle accidents. Driver monitoring systems, utilizing a range of sensors and techniques, offer an effective method to monitor and alert drivers to minimize driver error and reduce risky driving behaviors, thus helping to avoid Safety Critical Events (SCEs) and enhance overall driving safety. Artificial Intelligence (AI) tools, in particular, have been widely investigated to improve the efficiency and accuracy of driver monitoring or analysis of SCEs. To better understand the state-of-the-art practices and potential directions for AI tools in this domain, this work is an inaugural attempt to consolidate AI-related tools from academic and industry perspectives. We include an extensive review of AI models and sensors used in driver gaze analysis, driver state monitoring, and analyzing SCEs. Furthermore, researchers identified essential AI tools, both in academia and industry, utilized for camera-based driver monitoring and SCE analysis, in the market. Recommendations for future research directions are presented based on the identified tools and the discrepancies between academia and industry in previous studies. This effort provides a valuable resource for researchers and practitioners seeking a deeper understanding of leveraging AI tools to minimize driver errors, avoid SCEs, and increase driving safety.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Towards Cooperative Flight Control Using Visual-Attention

Lianhao Yin,Makram Chahine,Tsun-Hsuan Wang,Tim Seyde,Chao Liu,Mathias Lechner,Ramin Hasani,Daniela Rus

2023-09-21

Abstract:The cooperation of a human pilot with an autonomous agent during flight control realizes parallel autonomy. We propose an air-guardian system that facilitates cooperation between a pilot with eye tracking and a parallel end-to-end neural control system. Our vision-based air-guardian system combines a causal continuous-depth neural network model with a cooperation layer to enable parallel autonomy between a pilot and a control system based on perceived differences in their attention profiles. The attention profiles for neural networks are obtained by computing the networks' saliency maps (feature importance) through the VisualBackProp algorithm, while the attention profiles for humans are either obtained by eye tracking of human pilots or saliency maps of networks trained to imitate human pilots. When the attention profile of the pilot and guardian agents align, the pilot makes control decisions. Otherwise, the air-guardian makes interventions and takes over the control of the aircraft. We show that our attention-based air-guardian system can balance the trade-off between its level of involvement in the flight and the pilot's expertise and attention. The guardian system is particularly effective in situations where the pilot was distracted due to information overload. We demonstrate the effectiveness of our method for navigating flight scenarios in simulation with a fixed-wing aircraft and on hardware with a quadrotor platform.

Robotics,Artificial Intelligence
Machine Vision and Machine Learning for Intelligent Agrobots: A review

D Bini,D Pamela,Shajin Prince,Bini D,Pamela D

DOI: https://doi.org/10.1109/icdcs48716.2020.243538

2020-03-01

Abstract:An intelligent precise autonomous farming by an agricultural robot achieves the farm duties possibly harvesting, weed detection, disease identification, pruning and fertilizing deals with path planning and mapping of the unstructured and uncertain environment. A machine vision-based Agrobots along with artificial intelligence provides unmanned ground vehicle and unmanned aerial vehicle to navigate the path and to implement the agricultural task for minimizing labour and increasing quality food production. The perception-related work uses a machine learning algorithm to detect the feature and analyze the agricultural tasks for the autonomous machine. The trained data sets create the ability for robots to learn and decide the farm practices. The dawn of autonomous system design gives us the outlook to develop a wide range of flexible agronomic equipment based on multi-robot, smart machines and human-robot systems which lessen waste, progresses economic feasibility also reduces conservational impact and intensifies food sustainability. The multi-tasking Agrobots overcomes the effort of farmers in agricultural husbandry, independent of the climatic conditions. In this paper, a study on Agrobots effective in a diverse environment, its control and action process conjoined with mapping and detection using machine vision and machine learning algorithms are distinguished.
Vision-based Driver Assistance Systems: Survey, Taxonomy and Advances

Jonathan Horgan,Ciarán Hughes,John McDonald,Senthil Yogamani

DOI: https://doi.org/10.1109/ITSC.2015.329

2021-04-26

Abstract:Vision-based driver assistance systems is one of the rapidly growing research areas of ITS, due to various factors such as the increased level of safety requirements in automotive, computational power in embedded systems, and desire to get closer to autonomous driving. It is a cross disciplinary area encompassing specialised fields like computer vision, machine learning, robotic navigation, embedded systems, automotive electronics and safety critical software. In this paper, we survey the list of vision based advanced driver assistance systems with a consistent terminology and propose a taxonomy. We also propose an abstract model in an attempt to formalize a top-down view of application development to scale towards autonomous driving system.

Robotics,Computer Vision and Pattern Recognition
Computer Vision-Based Human-Machine Collision Warning System on Construction Site

Botao Gu,Hongling Guo,Yuecheng Huang,Huey Wen Lim,Dongping Fang

DOI: https://doi.org/10.1088/1755-1315/1101/3/032014

2022-01-01

IOP Conference Series Earth and Environmental Science

Abstract:With the increase of mechanization in the construction industry, more and more construction machines appear on construction sites. This improves construction efficiency but leads to the high risk of human-machine collision. At present, human-machine collision risks are usually monitored and warned manually. It is difficult to identify relevant risks in time due to the lack of sufficient safety supervisors. This paper proposes an efficient human-machine collision warning system based on computer vision and deep learning. The designed system and relevant devices are installed on construction machinery. It can quickly identify workers and measure the distance between workers and machinery in the complex and dynamic environment so as to quickly warn the risk of human-machine collision based on a threshold. Power supply restriction and harsh environment such as dust and vibration are also considered to ensure the stable and effective operation of the system. The system was tested on a construction site for 6 weeks. The result shows that the system has a significant positive effect on the early warning of human-machine collision risk, thus benefiting the improvement of safety performance of workers in human-machine collaboration scenes.
Hardware Accelerators for Autonomous Cars: A Review

Ruba Islayem,Fatima Alhosani,Raghad Hashem,Afra Alzaabi,Mahmoud Meribout

2024-04-26

Abstract:Autonomous Vehicles (AVs) redefine transportation with sophisticated technology, integrating sensors, cameras, and intricate algorithms. Implementing machine learning in AV perception demands robust hardware accelerators to achieve real-time performance at reasonable power consumption and footprint. Lot of research and development efforts using different technologies are still being conducted to achieve the goal of getting a fully AV and some cars manufactures offer commercially available systems. Unfortunately, they still lack reliability because of the repeated accidents they have encountered such as the recent one which happened in California and for which the Cruise company had its license suspended by the state of California for an undetermined period [1]. This paper critically reviews the most recent findings of machine vision systems used in AVs from both hardware and algorithmic points of view. It discusses the technologies used in commercial cars with their pros and cons and suggests possible ways forward. Thus, the paper can be a tangible reference for researchers who have the opportunity to get involved in designing machine vision systems targeting AV

Hardware Architecture,Robotics

Machine Vision

Traffic Collisions Early Warning Aided by Small Unmanned Aerial Vehicle Companion

Enhancing Situational Awareness of Helicopter Pilots in Unmanned Aerial Vehicle-Congested Environments Using an Airborne Visual Artificial Intelligence Approach

The Role of Machine Vision for Intelligent Vehicles

Machine vision-based autonomous road hazard avoidance system for self-driving vehicles

Artificial Intelligence in Adaptive In-Cab Interfaces

Machine Vision for UAS Ground Operations

Design of Autonomous Obstacle Avoidance System for Driverless Vehicle Based on Machine Vision

The Machine Vision Iceberg Explained: Advancing Dynamic Testing by Considering Holistic Environmental Relations

Applications of Computer Vision in Autonomous Vehicles: Methods, Challenges and Future Directions

IVVI: Intelligent vehicle based on visual information

The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond

Survey on Human-Vehicle Interactions and AI Collaboration for Optimal Decision-Making in Automated Driving

Vision-based intelligent vehicles: State of the art and perspectives

Machine visions: A corporate imaginary of artificial sight

Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles

Towards Cooperative Flight Control Using Visual-Attention

Machine Vision and Machine Learning for Intelligent Agrobots: A review

Vision-based Driver Assistance Systems: Survey, Taxonomy and Advances

Computer Vision-Based Human-Machine Collision Warning System on Construction Site

Hardware Accelerators for Autonomous Cars: A Review