Abstract:The Ocean Aware project, led by Innovasea and funded through Canada's Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags. This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation. Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras. This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites. Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras. To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters. First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA. Although there has been extensive research in the literature identifying particular fish such as eel vs. non-eel and seal vs. fish, to our knowledge this is the first successful application of deep learning for classifying multiple fish species with high resolution imaging sonar. Second, we integrated the Norfair object tracking framework to track and count fish using a public video dataset captured by optical cameras from the Wells Dam fish ladder on the Columbia River in Washington State, USA. Our results demonstrate that deep learning models can indeed be used to detect, classify species, and track fish using both high resolution imaging sonar and underwater video from a fish ladder. This work is a first step toward developing a fully implemented system which can accurately detect, classify and generate insights about fish in a wide variety of fish passage environments and conditions with data collected from multiple types of sensors.

Fish-lateral-inspired Pressure Sensing Neural Networks for Underwater Object Identification

Bionic Artificial Lateral Line Underwater Localization Based on the Neural Network Method

Online Fish Tracking with Portable Smart Device for Ocean Observatory Network

Feature Fusion of Body Surface and Motion-Based Instance Segmentation for High-Density Fish in Industrial Aquaculture

Embedded Online Fish Detection and Tracking System Via YOLOv3 and Parallel Correlation Filter

Hydrodynamic object identification with artificial neural models

Deep learning model inspired by lateral line system for underwater object detection

A Novel AMSS-FFN for Underwater Multisource Localization Using Artificial Lateral Line.

Reconstructing the pressure field around swimming fish using a physics-informed neural network

Underwater Geometric Target Recognition Based on Information Fusion of Active Electric Field and Pressure Sensors

Unsteady Cavity Pressure Distribution Recovery for Underwater Axisymmetric Body Via Deep Learning

Underwater Positioning Based on an Artificial Lateral Line and a Generalized Regression Neural Network

Underwater Source Localization Using an Artificial Lateral Line System With Pressure and Flow Velocity Sensor Fusion

A Pressure Sensory System Inspired by the Fish Lateral Line: Hydrodynamic Force Estimation and Wall Detection

A Highly Sensitive Deep-Sea Hydrodynamic Pressure Sensor Inspired by Fish Lateral Line

Simulation Research in Water Condition Recognition Method Based on Fish Lateral Line Sensing Mechanism

Design and development of autonomous robotic fish for object detection and tracking

Underwater Biomimetic Lateral Line Sensor Based on Triboelectric Nanogenerator for Dynamic Pressure Monitoring and Trajectory Perception

Enhanced Hydrofoil Pressure Field Reconstruction Through Phase-Informed Sensing and Sensor Optimization

Study on Swimming Modes Sensing of Robotic Fish Based on Artificial Lateral Line System

Automated Detection, Classification and Counting of Fish in Fish Passages With Deep Learning