Abstract:Fish are capable of learning complex relations found in their surroundings, and harnessing their knowledge may help to improve the autonomy and adaptability of robots. Here, we propose a novel learning from demonstration framework to generate fish-inspired robot control programs with as little human intervention as possible. The framework consists of six core modules: (1) task demonstration, (2) fish tracking, (3) analysis of fish trajectories, (4) acquisition of robot training data, (5) generating a perception-action controller, and (6) performance evaluation. We first describe these modules and highlight the key challenges pertaining to each one. We then present an artificial neural network for automatic fish tracking. The network detected fish successfully in 85% of the frames, and in these frames, its average pose estimation error was less than 0.04 body lengths. We finally demonstrate how the framework works through a case study focusing on a cue-based navigation task. Two low-level perception-action controllers were generated through the framework. Their performance was measured using two-dimensional particle simulations and compared against two benchmark controllers, which were programmed manually by a researcher. The fish-inspired controllers had excellent performance when the robot was started from the initial conditions used in fish demonstrations (>96% success rate), outperforming the benchmark controllers by at least 3%. One of them also had an excellent generalisation performance when the robot was started from random initial conditions covering a wider range of starting positions and heading angles (>98% success rate), again outperforming the benchmark controllers by 12%. The positive results highlight the utility of the framework as a research tool to form biological hypotheses on how fish navigate in complex environments and design better robot controllers on the basis of biological findings.

Zebrafish (Danio rerio) behavioural response to bioinspired robotic fish and mosquitofish (Gambusia affinis)

Influence of robotic shoal size, configuration, and activity on zebrafish behavior in a free-swimming environment

Infiltrating the zebrafish swarm: design, implementation and experimental tests of a miniature robotic fish lure for fish–robot interaction studies

Closed-loop control of zebrafish behaviour in three dimensions using a robotic stimulus

How to Blend a Robot within a Group of Zebrafish: Achieving Social Acceptance through Real-time Calibration of a Multi-level Behavioural Model

Evolutionary optimisation of neural network models for fish collective behaviours in mixed groups of robots and zebrafish

Zebrafish Case

Quantifying the biomimicry gap in biohybrid robot-fish pairs

Fish and robots swimming together: attraction towards the robot demands biomimetic locomotion

Fish and Robots Swimming Together in a Water Tunnel: Robot Color and Tail-Beat Frequency Influence Fish Behavior

Robot Programming from Fish Demonstrations

Zebrafish automatic monitoring system for conditioning and behavioral analysis

Fish and robot dancing together: bluefin killifish females respond differently to the courtship of a robot with varying color morphs

Altering Fish Behavior by Sensing Swarm Patterns of Fish in an Artificial Aquatic Environment Using an Interactive Robotic Fish

Fish Can Save Energy Via Proprioceptive Sensing.

Proactivity of fish and leadership of self-propelled robotic fish during interaction

GoFish: a foray into open-source, aquatic behavioral automation

Zebrafish collective behaviour in heterogeneous environment modeled by a stochastic model based on visual perception

An Open-Source Programmable Interface for Sensory–Motor Biotests with Zebrafish (Danio rerio)

Real-Time Feedback-Controlled Robotic Fish for Behavioral Experiments With Fish Schools