Abstract:The Ocean Aware project, led by Innovasea and funded through Canada's Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags. This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation. Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras. This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites. Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras. To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters. First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA. Although there has been extensive research in the literature identifying particular fish such as eel vs. non-eel and seal vs. fish, to our knowledge this is the first successful application of deep learning for classifying multiple fish species with high resolution imaging sonar. Second, we integrated the Norfair object tracking framework to track and count fish using a public video dataset captured by optical cameras from the Wells Dam fish ladder on the Columbia River in Washington State, USA. Our results demonstrate that deep learning models can indeed be used to detect, classify species, and track fish using both high resolution imaging sonar and underwater video from a fish ladder. This work is a first step toward developing a fully implemented system which can accurately detect, classify and generate insights about fish in a wide variety of fish passage environments and conditions with data collected from multiple types of sensors.

Probabilistic Model-Based Active Learning with Attention Mechanism for Fish Species Recognition

Accurate Fish Detection under Marine Background Noise Based on the Retinex Enhancement Algorithm and CNN

Multi-class Fish Stock Statistics Technology Based on Object Classification and Tracking Algorithm

Semi-supervised learning-based live fish identification in aquaculture using modified deep convolutional generative adversarial networks

Accelerating Species Recognition and Labelling of Fish From Underwater Video With Machine-Assisted Deep Learning

Class-Aware Fish Species Recognition Using Deep Learning for an Imbalanced Dataset

Fish Species Classification Using Deep Learning and Appearance-Based Feature Extraction

Fish species classification in unconstrained underwater environments based on deep learning

Automated Detection, Classification and Counting of Fish in Fish Passages With Deep Learning

A Deep Learning-Based Lightweight Model for the Detection of Marine Fishes

Fish detection and species classification in underwater environments using deep learning with temporal information

Computer-Simulated Virtual Image Datasets to Train Machine Learning Models for Non-Invasive Fish Detection in Recirculating Aquaculture

Deepwater-SE: A new automatic fish classification network based on deep learning

A Deep CNN-Based Salinity and Freshwater Fish Identification and Classification Using Deep Learning and Machine Learning

Deep learning with self-supervision and uncertainty regularization to count fish in underwater images

Marine Fish Object Detection Based on YOLOv5 and Attention Mechanism.

An Intelligent Deep Learning Enabled Marine Fish Species Detection and Classification Model

A recurrent network based on active learning for the assessment of fish feeding status

Computer vision in aquaculture: a case study of juvenile fish counting

Transfer Learning and SE-ResNet152 Networks-Based for Small-Scale Unbalanced Fish Species Identification.

Real-Time Underwater Fish Detection and Recognition Based on CBAM-YOLO Network with Lightweight Design