Don’t Get Lost in the Crowd: Graph Convolutional Network for Online Animal Tracking in Dense Groups

Andrea Parmiggiani,Dong Liu,Eric Psota,Robert Fitzgerald,Tomas Norton

DOI: https://doi.org/10.1016/j.compag.2023.108038

IF: 8.3

2023-01-01

Computers and Electronics in Agriculture

Abstract:Tracking the behaviour of animals in group-housed situations is a critical area of study for precision livestock farming, but it poses challenges in diverse and crowded environments. Existing methods often struggle with false positives and false negatives due to the complexities of these scenarios. In this study, we propose a robust computer vision algorithm for long-term (>10mins) animal tracking, with a primary focus on group-housed pigs. Our method addresses the limitations of current approaches by effectively handling errors from the detector in challenging environments. Our approach integrates Graph Convolutional Networks.(GCNs) with deep learning-based object detection techniques. We represent the data as a graph structure, with nodes corresponding to multiple animal detections over several frames. Edges connect the detections that do not appear in the same frame. The model's objective is to perform edge classification, where each edge is associated with a scalar representing the probability of being the same object. To enhance the robustness of edge classification, we combine the classifier with an ensemble predictor, enabling accurate tracking of animal identities for extended periods. Notably, our graph convolutional model achieves accurate re identification of non-consecutive animal detections in real-time without the need for trajectory prediction or ID association. We evaluate our method by comparing its performance against other tracking methods, including DeepSORT-a state-of-the-art animal tracking method that relies on a Kalman filter, a deep appearance descriptor, and the Hungarian algorithm for trajectory prediction and ID association. Our comparison demonstrates the advantages of our method, as it (1) improves the IDF1 score, indicating enhanced ID association accuracy, by 1.72%; (2) mitigates 93% of errors resulting from ID-switch and deviation; (3) extends the tracking duration of each animal by approximately 66% in challenging conditions; and (4) achieves a processing speed that is 1000 times faster than DeepSORT when operating at 22fps with the detector. The potential applications of our method extend to various aspects of livestock management. By accurately tracking individual animals, we can monitor behaviours such as feeding, drinking, and aggressive interactions. This information can lead to improvements in animal welfare, resource allocation, and breeding strategies. Follow-ups of this research are available at: https://gitlab.kuleuven.be/m3-biores/public/m3pig.