Visual seafloor mapping with autonomous robots

Amos Matsiko

DOI: https://doi.org/10.1126/scirobotics.ads9444

2024-09-11

Abstract:Autonomous robots adopt navigation-aided hierarchical reconstruction to visually map the seafloor.

What problem does this paper attempt to address?

Semihierarchical Reconstruction and Weak-area Revisiting for Robotic Visual Seafloor Mapping

Mengkun She,Yifan Song,David Nakath,Kevin Köser

2024-06-29

Abstract:Despite impressive results achieved by many on-land visual mapping algorithms in the recent decades, transferring these methods from land to the deep sea remains a challenge due to harsh environmental conditions. Images captured by autonomous underwater vehicles (AUVs), equipped with high-resolution cameras and artificial illumination systems, often suffer from heterogeneous illumination and quality degradation caused by attenuation and scattering, on top of refraction of light rays. These challenges often result in the failure of on-land SLAM approaches when applied underwater or cause SfM approaches to exhibit drifting or omit challenging images. Consequently, this leads to gaps, jumps, or weakly reconstructed areas. In this work, we present a navigation-aided hierarchical reconstruction approach to facilitate the automated robotic 3D reconstruction of hectares of seafloor. Our hierarchical approach combines the advantages of SLAM and global SfM that is much more efficient than incremental SfM, while ensuring the completeness and consistency of the global map. This is achieved through identifying and revisiting problematic or weakly reconstructed areas, avoiding to omit images and making better use of limited dive time. The proposed system has been extensively tested and evaluated during several research cruises, demonstrating its robustness and practicality in real-world conditions.

Computer Vision and Pattern Recognition
Autonomous Exploration and General Visual Inspection of Ship Ballast Water Tanks using Aerial Robots

Mihir Dharmadhikari,Paolo De Petris,Mihir Kulkarni,Nikhil Khedekar,Huan Nguyen,Arnt Erik Stene,Eivind Sjøvold,Kristian Solheim,Bente Gussiaas,Kostas Alexis

2023-11-07

Abstract:This paper presents a solution for the autonomous exploration and inspection of Ballast Water Tanks (BWTs) of marine vessels using aerial robots. Ballast tank compartments are critical for a vessel's safety and correspond to confined environments often connected through particularly narrow manholes. The method enables their volumetric exploration combined with visual inspection subject to constraints regarding the viewing distance from a surface. We present evaluation studies in simulation, in a mission consisting of 18 BWT compartments, and in 3 field experiments inside real vessels. The data from one of the experiments is also post-processed to generate semantically-segmented meshes of inspection-important geometries. Geometric models can be associated with onboard camera images for detailed and intuitive analysis.

Robotics
Pose‐graph underwater simultaneous localization and mapping for autonomous monitoring and 3D reconstruction by means of optical and acoustic sensors

Alessandro Bucci,Alessandro Ridolfi,Benedetto Allotta

DOI: https://doi.org/10.1002/rob.22375

IF: 6.385

2024-06-12

Journal of Field Robotics

Abstract:Modern mobile robots require precise and robust localization and navigation systems to achieve mission tasks correctly. In particular, in the underwater environment, where Global Navigation Satellite Systems cannot be exploited, the development of localization and navigation strategies becomes more challenging. Maximum A Posteriori (MAP) strategies have been analyzed and tested to increase navigation accuracy and take into account the entire history of the system state. In particular, a sensor fusion algorithm relying on a MAP technique for Simultaneous Localization and Mapping (SLAM) has been developed to fuse information coming from a monocular camera and a Doppler Velocity Log (DVL) and to consider the landmark points in the navigation framework. The proposed approach can guarantee to simultaneously locate the vehicle and map the surrounding environment with the information extracted from the images acquired by a bottom‐looking optical camera. Optical sensors can provide constraints between the vehicle poses and the landmarks belonging to the observed scene. The DVL measurements have been employed to solve the unknown scale factor and to guarantee the correct vehicle localization even in the absence of visual features. Furthermore, to evaluate the mapping capabilities of the SLAM algorithm, the obtained point cloud is elaborated with a Poisson reconstruction method to obtain a smooth seabed surface. After validating the proposed solution through realistic simulations, an experimental campaign at sea was conducted in Stromboli Island (Messina), Italy, where both the navigation and the mapping performance have been evaluated.

robotics
Camera-based mapping in search-and-rescue via flying and ground robot teams

Jamshidnejad, Anahita

DOI: https://doi.org/10.1007/s00138-024-01594-4

IF: 2.983

2024-08-22

Machine Vision and Applications

Abstract:Search and rescue (SaR) is challenging, due to the unknown environmental situation after disasters occur. Robotics has become indispensable for precise mapping of the environment and for locating the victims. Combining flying and ground robots more effectively serves this purpose, due to their complementary features in terms of viewpoint and maneuvering. To this end, a novel, cost-effective framework for mapping unknown environments is introduced that leverages You Only Look Once and video streams transmitted by a ground and a flying robot. The integrated mapping approach is for performing three crucial SaR tasks: localizing the victims, i.e., determining their position in the environment and their body pose, tracking the moving victims, and providing a map of the ground elevation that assists both the ground robot and the SaR crew in navigating the SaR environment. In real-life experiments at the CyberZoo of the Delft University of Technology, the framework proved very effective and precise for all these tasks, particularly in occluded and complex environments.

computer science, cybernetics, artificial intelligence,engineering, electrical & electronic
Robotic Exploration for Mapping

Akanimoh Adeleye

DOI: https://doi.org/10.48550/arXiv.2307.08690

2023-07-18

Abstract:Robotic Exploration has evolved rapidly in the past two decades as new and more complex techniques have been created to explore unknown regions efficiently. Exciting advancements in exploration, autonomous navigation, and sensor technology have created opportunities for robots to be utilized in new environments and for new objectives ranging from mapping of abandon mines and deep oceans to the efficient creation of indoor models for navigation and search. In this paper we present and discuss a number of examples in research literature of these recent advancements, specifically focusing on robotic exploration algorithms for unmanned vehicles.

Robotics
Topological Navigation for Autonomous Underwater Vehicles in Confined Semi-Structured Environments

Claudio Rossi,Adrian Caro Zapata,Zorana Milosevic,Ramon Suarez,Sergio Dominguez

DOI: https://doi.org/10.3390/s23052371

2023-02-21

Abstract:In this work, we present the design, implementation, and simulation of a topology-based navigation system for the UX-series robots, a spherical underwater vehicle designed to explore and map flooded underground mines. The objective of the robot is to navigate autonomously in the 3D network of tunnels of a semi-structured but unknown environment in order to gather geoscientific data. We start from the assumption that a topological map has been generated by a low-level perception and SLAM module in the form of a labeled graph. However, the map is subject to uncertainties and reconstruction errors that the navigation system must address. First, a distance metric is defined to compute node-matching operations. This metric is then used to enable the robot to find its position on the map and navigate it. To assess the effectiveness of the proposed approach, extensive simulations have been carried out with different randomly generated topologies and various noise rates.
Robot Goes Fishing: Rapid, High-Resolution Biological Hotspot Mapping in Coral Reefs with Vision-Guided Autonomous Underwater Vehicles

Daniel Yang,Levi Cai,Stewart Jamieson,Yogesh Girdhar

2024-02-01

Abstract:Coral reefs are fast-changing and complex ecosystems that are crucial to monitor and study. Biological hotspot detection can help coral reef managers prioritize limited resources for monitoring and intervention tasks. Here, we explore the use of autonomous underwater vehicles (AUVs) with cameras, coupled with visual detectors and photogrammetry, to map and identify these hotspots. This approach can provide high spatial resolution information in fast feedback cycles. To the best of our knowledge, we present one of the first attempts at using an AUV to gather visually-observed, fine-grain biological hotspot maps in concert with topography of a coral reefs. Our hotspot maps correlate with rugosity, an established proxy metric for coral reef biodiversity and abundance, as well as with our visual inspections of the 3D reconstruction. We also investigate issues of scaling this approach when applied to new reefs by using these visual detectors pre-trained on large public datasets.

Robotics,Computer Vision and Pattern Recognition
Structured Light-Based Underwater Collision-Free Navigation and Dense Mapping System for Refined Exploration in Unknown Dark Environments

Yaming Ou,Junfeng Fan,Chao Zhou,Song Kang,Zhuoliang Zhang,Zeng-Guang Hou,Min Tan

DOI: https://doi.org/10.1109/tsmc.2024.3370917

2024-01-01

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Abstract:Underwater collision-free navigation and dense reconstruction are essential for marine refined exploration. However, existing passive vision-based methods are difficult to apply in low-light and weak-feature underwater environments. In this article, a more adaptable three-dimensional (3-D) dense mapping robotic system based on self-designed scanning binocular structured light (BSL), named ROV-Scanner, is developed to address this challenge. First, the measurement principle based on the refraction model ensures its high accuracy. Second, an underwater 3-D dense mapping algorithm fusing the Doppler velocity log (DVL), inertial measurement unit (IMU), and pressure sensor multifrequency information is proposed to realize dense mapping during robot motion. Then, an air–water two-stage extrinsic calibration algorithm is proposed. In particular, the extrinsic parameters between DVL and camera are innovatively calibrated using BSL, enhancing robustness. Furthermore, for the first time, a framework of BSL-based collision-free navigation is presented to guarantee the safe movement of the system in unknown environments. Experimental results show that our system can simultaneously achieve autonomous collision-free navigation and dense mapping in dark underwater environments, which has great potential for application in marine refined exploration.

automation & control systems,computer science, cybernetics
Robust Perception and Navigation of Autonomous Surface Vehicles in Challenging Environments

Mingi Jeong

2024-05-28

Abstract:Research on coastal regions traditionally involves methods like manual sampling, monitoring buoys, and remote sensing, but these methods face challenges in spatially and temporally diverse regions of interest. Autonomous surface vehicles (ASVs) with artificial intelligence (AI) are being explored, and recognized by the International Maritime Organization (IMO) as vital for future ecosystem understanding. However, there is not yet a mature technology for autonomous environmental monitoring due to typically complex coastal situations: (1) many static (e.g., buoy, dock) and dynamic (e.g., boats) obstacles not compliant with the rules of the road (COLREGs); (2) uncharted or uncertain information (e.g., non-updated nautical chart); and (3) high-cost ASVs not accessible to the community and citizen science while resulting in technology illiteracy. To address the above challenges, my research involves both system and algorithmic development: (1) a robotic boat system for stable and reliable in-water monitoring, (2) maritime perception to detect and track obstacles (such as buoys, and boats), and (3) navigational decision-making with multiple-obstacle avoidance and multi-objective optimization.

Robotics
Opportunistic (Re)planning for Long-Term Deep-Ocean Inspection: An Autonomous Underwater Architecture

Elisa Tosello,Paolo Bonel,Alberto Buranello,Marco Carraro,Alessandro Cimatti,Lorenzo Granelli,Stefan Panjkovic,Andrea Micheli

DOI: https://doi.org/10.1109/mra.2024.3352810

IF: 5.229

2024-03-19

IEEE Robotics & Automation Magazine

Abstract:Robots are increasingly used in subsea environments because of their positive impact on human safety and operational capabilities in the deep ocean. However, achieving full autonomy remains challenging because of the extreme conditions they encounter. In this context, we propose an autonomous underwater architecture for long-term deep-ocean inspection that robustly plans activities and efficiently deliberates with no human help. It combines the innovative Saipem's Hydrone-R subsea vehicle with an advanced planning architecture, resulting in a robot that autonomously perceives its surroundings, plans a mission, and adapts in real time to contingencies and opportunities. After describing the robot hardware, we present the technological advancements achieved in building its software, along with several compelling use cases. We explore scenarios where the robot conducts long-term underwater missions operating under resource constraints while remaining responsive to opportunities, such as new inspection points. Our solution gained significant reliability and acceptance within the oil and gas community as evidenced by its current deployment on a real field in Norway.

robotics,automation & control systems
Real-Time Stereo-Based Ocean Surface Mapping for Robotic Floating Platforms: Concept and Methodology

Or Greenberg,Boaz Ben-Moshe

DOI: https://doi.org/10.3390/s23083857

2023-04-10

Abstract:Consider the case of a small, unmanned boat that is performing an autonomous mission. Naturally, such a platform might need to approximate the ocean surface of its surroundings in real-time. Much like obstacle mapping in autonomous (off-road) rovers, an approximation of the ocean surface in a vessel's surroundings in real-time can be used for improved control and optimized route planning. Unfortunately, such an approximation seems to require either expensive and heavy sensors or external logistics that are mostly not available for small or low-cost vessels. In this paper, we present a real-time method for detecting and tracking ocean waves around a floating object that is based on stereo vision sensors. Based on a large set of experiments, we conclude that the presented method allows reliable, real-time, and cost-effective ocean surface mapping suitable for small autonomous boats.
Deep Learning for Visual Navigation of Underwater Robots

M. Sunbeam

2023-10-30

Abstract:This paper aims to briefly survey deep learning methods for visual navigation of underwater robotics. The scope of this paper includes the visual perception of underwater robotics with deep learning methods, the available visual underwater datasets, imitation learning, and reinforcement learning methods for navigation. Additionally, relevant works will be categorized under the imitation learning or deep learning paradigm for underwater robots for clarity of the training methodologies in the current landscape. Literature that uses deep learning algorithms to process non-visual data for underwater navigation will not be considered, except as contrasting examples.

Robotics,Computer Vision and Pattern Recognition,Machine Learning
Marine Robots: From Laboratories to the Real Underwater Adventure

Chen Wang,Ming Cao,Junzhi Yu,Guangming Xie,Derek A. Paley,Corina Barbalata,Stefan B. Williams

DOI: https://doi.org/10.1109/MRA.2024.3353628

IF: 5.229

2024-01-01

IEEE Robotics & Automation Magazine

Abstract:In our continuous exploration and exploitation of the ocean, marine robots are powerful tools for accessing oceanic environments that are sometimes harsh or even too dangerous for humans. Early marine robots acted under human supervision, executing preprogrammed tasks or working under the control of highly trained specialists. In recent years, with the advancements in hardware and software, marine robots have begun to possess the capability to autonomously sense and adapt to the intricate ocean environment, thus working more efficiently and intelligently. By integrating recent advances in science and technology, ranging from biomimetic technology and new materials to advanced artificial intelligence methods, marine robots have the potential to be more powerful in the exploration of the ocean environment and facilitating a broader spectrum of tasks, such as studying marine life in their natural habitats or performing inspections and dexterous manipulation in the deep sea.
NeuroSwarm: Multi-Agent Neural 3D Scene Reconstruction and Segmentation with UAV for Optimal Navigation of Quadruped Robot

Iana Zhura,Denis Davletshin,Nipun Dhananjaya Weerakkodi Mudalige,Aleksey Fedoseev,Robinroy Peter,Dzmitry Tsetserukou

2023-08-03

Abstract:Quadruped robots have the distinct ability to adapt their body and step height to navigate through cluttered environments. Nonetheless, for these robots to utilize their full potential in real-world scenarios, they require awareness of their environment and obstacle geometry. We propose a novel multi-agent robotic system that incorporates cutting-edge technologies. The proposed solution features a 3D neural reconstruction algorithm that enables navigation of a quadruped robot in both static and semi-static environments. The prior areas of the environment are also segmented according to the quadruped robots' abilities to pass them. Moreover, we have developed an adaptive neural field optimal motion planner (ANFOMP) that considers both collision probability and obstacle height in 2D space.Our new navigation and mapping approach enables quadruped robots to adjust their height and behavior to navigate under arches and push through obstacles with smaller dimensions. The multi-agent mapping operation has proven to be highly accurate, with an obstacle reconstruction precision of 82%. Moreover, the quadruped robot can navigate with 3D obstacle information and the ANFOMP system, resulting in a 33.3% reduction in path length and a 70% reduction in navigation time.

Robotics
Towards Automated Sample Collection and Return in Extreme Underwater Environments

Gideon Billings,Matthew Walter,Oscar Pizarro,Matthew Johnson-Roberson,Richard Camilli

DOI: https://doi.org/10.55417/fr.2022045

2022-03-10

Field Robotics

Abstract:In this report, we present the system design, operational strategy, and results of coordinated multivehicle field demonstrations of autonomous marine robotic technologies in searchfor- life missions within the Pacific shelf margin of Costa Rica and the Santorini-Kolumbo caldera complex, which serve as analogs to environments that may exist in oceans beyond Earth. This report focuses on the automation of remotely operated vehicle (ROV) manipulator operations for targeted biological sample-collection-and-return from the seafloor. In the context of future extraterrestrial exploration missions to ocean worlds, an ROV is an analog to a planetary lander, which must be capable of high-level autonomy. Our field trials involve two underwater vehicles, the SuBastian ROV and the Nereid Under Ice (NUI) hybrid ROV for mixed initiative (i.e., teleoperated or autonomous) missions, both equipped seven-degrees-of-freedom hydraulic manipulators.We describe an adaptable, hardware-independent computer vision architecture that enables high-level automated manipulation. The vision system provides a three-dimensional understanding of the workspace to inform manipulator motion planning in complex unstructured environments. We demonstrate the effectiveness of the vision system and control framework through field trials in increasingly challenging environments, including the automated collection and return of biological samples from within the active undersea volcano Kolumbo. Based on our experiences in the field, we discuss the performance of our system and identify promising directions for future research.
Robot Navigation in Unseen Spaces using an Abstract Map

Ben Talbot,Feras Dayoub,Peter Corke,Gordon Wyeth

DOI: https://doi.org/10.1109/TCDS.2020.2993855

2020-05-15

Abstract:Human navigation in built environments depends on symbolic spatial information which has unrealised potential to enhance robot navigation capabilities. Information sources such as labels, signs, maps, planners, spoken directions, and navigational gestures communicate a wealth of spatial information to the navigators of built environments; a wealth of information that robots typically ignore. We present a robot navigation system that uses the same symbolic spatial information employed by humans to purposefully navigate in unseen built environments with a level of performance comparable to humans. The navigation system uses a novel data structure called the abstract map to imagine malleable spatial models for unseen spaces from spatial symbols. Sensorimotor perceptions from a robot are then employed to provide purposeful navigation to symbolic goal locations in the unseen environment. We show how a dynamic system can be used to create malleable spatial models for the abstract map, and provide an open source implementation to encourage future work in the area of symbolic navigation. Symbolic navigation performance of humans and a robot is evaluated in a real-world built environment. The paper concludes with a qualitative analysis of human navigation strategies, providing further insights into how the symbolic navigation capabilities of robots in unseen built environments can be improved in the future.

Robotics,Computer Vision and Pattern Recognition
Multirobot Multimodal Deep Sea Surveys: Use in Detailed Estimation of Manganese Crust Distribution

Umesh Neettiyath,Harumi Sugimatsu,Tetsu Koike,Kazunori Nagano,Tamaki Ura,Blair Thornton

DOI: https://doi.org/10.1109/mra.2023.3348304

IF: 5.229

2024-03-19

IEEE Robotics & Automation Magazine

Abstract:This article describes a multiyear survey of cobalt-rich manganese crust (Mn-crust) deposits using multiple underwater robots. Using two autonomous underwater vehicles and one remotely operated vehicle, mounted with camera systems, multibeam sonar, and subbottom sensors, large areas were surveyed by incorporating the advantages of each robot to create a comprehensive database of Mn-crust distribution estimates. The robots clocked in a total of 438 hours of seafloor observation, surveying about 589 km of seafloor in different locations. Specific use cases of the survey methodology and example results showing how each sensor contributes to the understanding of Mn-crust distribution are shown. The results from this survey can be combined with ship base multibeam data for seamount-scale estimates of Mn-crust volumetric distribution with high accuracy.

robotics,automation & control systems
Shoreline Mapping Using an Omni-directional Camera for Autonomous Surface Vehicle Applications

Anbumani Subicamanian,Xiaojin Gong,Jamie N. Riggins,Daniel J. Stilwell,Christopher L. Wyatt

DOI: https://doi.org/10.1109/oceans.2006.306906

2006-01-01

Abstract:Autonomous surface vehicles (ASVs) have the potential to operate for extended periods of time in coastal, estuarine, and riverine environments for a variety of scientific, environmental, and military applications. However, these environments are often highly dynamic and unstructured, and present new challenges for autonomous operations. Toward the goal of achieving truly autonomous long-term operations in highly unstructured maritime environments, we present a new approach to create a shoreline map with an ASV in real-time by combining the analysis of images from a single omni-directional camera and knowledge of the vehicle's location and motion information
Real-time Robust Mapping for an Autonomous Surface Vehicle Using an Omnidirectional Camera

Xiaojin Gong,Bin Xu,Caleb Reed,Chris Wyatt,Daniel Stilwell

DOI: https://doi.org/10.1109/wacv.2008.4544024

2008-01-01

Abstract:Towards the goal of achieving truly autonomous navigation for a surface vehicle in maritime environments, a critical task is to detect surrounding obstacles such as the shore, docks, and other boats. In this paper, we demonstrate a real-time vision-based mapping system which detects and localizes stationary obstacles using a single omnidirectional camera and navigational sensors (GPS and gyro). The main challenge of this work is to make mapping robust to a large number of outliers, which stem from waves and specular reflections on the surface of the water. To address this problem, a two-step robust outlier rejection method is proposed. Experimental results obtained in unstructured large-scale environments are presented and validated using topographic maps.
Large-Scale Dense 3D Mapping Using Submaps Derived From Orthogonal Imaging Sonars

John McConnell,Ivana Collado-Gonzalez,Paul Szenher,Brendan Englot

2024-12-05

Abstract:3D situational awareness is critical for any autonomous system. However, when operating underwater, environmental conditions often dictate the use of acoustic sensors. These acoustic sensors are plagued by high noise and a lack of 3D information in sonar imagery, motivating the use of an orthogonal pair of imaging sonars to recover 3D perceptual data. Thus far, mapping systems in this area only use a subset of the available data at discrete timesteps and rely on object-level prior information in the environment to develop high-coverage 3D maps. Moreover, simple repeating objects must be present to build high-coverage maps. In this work, we propose a submap-based mapping system integrated with a simultaneous localization and mapping (SLAM) system to produce dense, 3D maps of complex unknown environments with varying densities of simple repeating objects. We compare this submapping approach to our previous works in this area, analyzing simple and highly complex environments, such as submerged aircraft. We analyze the tradeoffs between a submapping-based approach and our previous work leveraging simple repeating objects. We show where each method is well-motivated and where they fall short. Importantly, our proposed use of submapping achieves an advance in underwater situational awareness with wide aperture multi-beam imaging sonar, moving toward generalized large-scale dense 3D mapping capability for fully unknown complex environments.

Robotics

Visual seafloor mapping with autonomous robots

Semihierarchical Reconstruction and Weak-area Revisiting for Robotic Visual Seafloor Mapping

Autonomous Exploration and General Visual Inspection of Ship Ballast Water Tanks using Aerial Robots

Pose‐graph underwater simultaneous localization and mapping for autonomous monitoring and 3D reconstruction by means of optical and acoustic sensors

Camera-based mapping in search-and-rescue via flying and ground robot teams

Robotic Exploration for Mapping

Topological Navigation for Autonomous Underwater Vehicles in Confined Semi-Structured Environments

Robot Goes Fishing: Rapid, High-Resolution Biological Hotspot Mapping in Coral Reefs with Vision-Guided Autonomous Underwater Vehicles

Structured Light-Based Underwater Collision-Free Navigation and Dense Mapping System for Refined Exploration in Unknown Dark Environments

Robust Perception and Navigation of Autonomous Surface Vehicles in Challenging Environments

Opportunistic (Re)planning for Long-Term Deep-Ocean Inspection: An Autonomous Underwater Architecture

Real-Time Stereo-Based Ocean Surface Mapping for Robotic Floating Platforms: Concept and Methodology

Deep Learning for Visual Navigation of Underwater Robots

Marine Robots: From Laboratories to the Real Underwater Adventure

NeuroSwarm: Multi-Agent Neural 3D Scene Reconstruction and Segmentation with UAV for Optimal Navigation of Quadruped Robot

Towards Automated Sample Collection and Return in Extreme Underwater Environments

Robot Navigation in Unseen Spaces using an Abstract Map

Multirobot Multimodal Deep Sea Surveys: Use in Detailed Estimation of Manganese Crust Distribution

Shoreline Mapping Using an Omni-directional Camera for Autonomous Surface Vehicle Applications

Real-time Robust Mapping for an Autonomous Surface Vehicle Using an Omnidirectional Camera

Large-Scale Dense 3D Mapping Using Submaps Derived From Orthogonal Imaging Sonars