Abstract:Tracking the fish is a key step in analyzing fish behavior, evaluating their health levels, and warning of abnormal water quality, so it is of significant importance for intelligent monitoring in fish farming. However, multi-object tracking for fish is a very challenging task due to foam occlusion in the factory tank, high individual similarity, and fast-paced motion of fish. To address the above issues, an online multi-fish tracking model called FishTrack is proposed with 3 branches of target detection, track prediction, and re-identification, which establishes the motion model and appearance model for fish simultaneously, thus achieving the online multi-fish tracking. It is a complete Encoder-Decoder structure and the Pyramid Vision Transformer (PVT) is adopted as the backbone network to extract multi-level features. Then an Encoder is specially designed to encode the historical information of the positions of fish, and automatically update its spatiotemporal information in an autoregressive fashion, to perform the fusion of the spatiotemporal information of fish targets and avoid manual selection of spatiotemporal features. Finally, a parallel dual-Decoder is used to decode the motion and appearance features of fish to reduce the interference of the two optimization directions during the joint model training. The motion cues are first used in linear assignment to reduce the interference of occlusion and deformation issues, and when the motion model fails to track, the appearance cues can recover the identity to deal with long occlusion. The experiments on the established multiple fish tracking dataset showed that the higher order tracking accuracy (HOTA) reached 71.4%, the multi-object tracking accuracy (MOTA) reached 94.8%, the most tracked ratio (MT) reached 93.3%, and the identification F1 score (IDF1) reached 82.5%. The results show that FishTrack can solve the problem of tracking accuracy decrease caused by foam occlusion and fish deformation, and save the inference time by sharing features, increasing the robustness of online multi-fish tracking in factory farming.

Real-Time Underwater Fish Tracking Based on Adaptive Multi-Appearance Model

A Robust Underwater Multiclass Fish-School Tracking Algorithm

Closed-Loop Tracking-by-Detection for ROV-Based Multiple Fish Tracking

Online Fish Tracking with Portable Smart Device for Ocean Observatory Network

Multi-class Fish Stock Statistics Technology Based on Object Classification and Tracking Algorithm

Fish Keypoints Detection for Ecology Monitoring Based on Underwater Visual Intelligence

Tracking Undulatory Body Motion Of Multiple Fish Based On Midline Dynamics Modeling

Fish Trajectory Extraction Based on Object Detection

FishTrack: Multi-object Tracking Method for Fish Using Spatiotemporal Information Fusion

Online 3D Behavioral Tracking of Aquatic Model Organism with a Dual-Camera System

Tracking Live Fish from Low-Contrast and Low-Frame-Rate Stereo Videos

Recent Advances of Target Tracking Applications in Aquaculture with Emphasis on Fish

An Effective and Robust Method for Tracking Multiple Fish in Video Image Based on Fish Head Detection

Underwater Fish Tracking for Moving Cameras based on Deformable Multiple Kernels

Visual Tracking of Multiple Robotic Fish for Cooperative Control

A Real Time Video Object Tracking Method for Fish

Development and Research of a Multi-Medium Motion Capture System for Underwater Intelligent Agents

Underwater Target Tracking Control of an Untethered Robotic Fish With a Camera Stabilizer

Underwater Long-Term Object Tracker for Marine Organism Capture

Consistent fish tracking via multiple underwater cameras

FishMOT: A Simple and Effective Method for Fish Tracking Based on IoU Matching