Abstract:Automated behavior analysis (ABA) strategies are being researched at a rapid rate to detect an array of behaviors across a range of species. There is growing optimism that soon ethologists will not have to manually decode hours (and hours) of animal behavior videos, but that instead computers will process them for us. However, before we assume ABA is ready for practical use, it is important to take a realistic look at exactly what ABA is being developed, the expertise being used to develop it, and the context in which these studies occur. Once we understand common pitfalls occurring during ABA development and identify limitations, we can construct robust ABA tools to achieve automated (ultimately even continuous and real time) analysis of behavioral data, allowing for more detailed or longer-term studies of behavior on larger numbers of animals than ever before. ABA is only as good as it is trained to be. A key starting point is having manually annotated data for model training and assessment. However, most ABA developers are not trained in ethology. Often no formal ethogram is developed and descriptions of target behaviors in ABA publications are limited or inaccurate. In addition, ABA is also frequently developed using small datasets, which lack sufficient variability in animal morphometrics, activities, camera viewpoints, and environmental features to be generalizable. Thus, ABA often needs to be further validated before being used satisfactorily on different populations or under other conditions, even for research purposes. Multidisciplinary teams of researchers including ethologists and ethicists as well as computer scientists, data scientists, and engineers are needed to help address problems when applying computer vision ABA to measure behavior. Reference datasets that can be used for behavior detection should be generated and shared that include image data, annotations, and baseline analyses for benchmarking. Also critical is the development of standards for creating such reference datasets and descriptions of best practices for methods for validating results from detection tools to ensure they are robust and generalizable. At present, only a handful of publicly available datasets exist that can be used for development of ABA tools. As we work to realize the promise of ABA (and subsequent precision livestock farming technologies) to detect animal behavior, a clear understanding of best practices, access to accurately annotated datasets, and networking among ethologists and ABA developers will increase our chances for rapid and robust successes.

Minimizing observer bias in animal behavior studies revisited: Improvement, but a long way to go

The hidden side of animal cognition research: Scientists' attitudes toward bias, replicability and scientific practice

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Changes in methodological study characteristics in psychology between 2010-2021

Fantastic beasts and how to study them: rethinking experimental animal behavior

Risk of bias reporting in the recent animal focal cerebral ischaemia literature

Transparency and reproducibility in the journal of contextual behavioral science: An audit study

Interobserver agreement and procedural fidelity: An odd asymmetry

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014–2017)

Common method biases in behavioral research: A critical review of the literature and recommended remedies.

Mitigating animal methods bias to reduce animal use and improve biomedical translation

The quest to develop automated systems for monitoring animal behavior

Recommendations to improve use and reporting of statistics in animal experiments

Animals, quality and the pursuit of relevance

Are We Becoming More Transparent? Survey Reporting Trends in Top Journals of Social Sciences

Improving reproducibility in animal research

The methodological quality of animal research in critical care: the public face of science

Experiences of early career researchers: Influences on the design and reporting of animal experiments, and the practical and emotional support needed to enhance best practice methods

Considering aspects of the 3Rs principles within experimental animal biology

Improving basic and translational science by accounting for litter-to-litter variation in animal models

Problems and Progress regarding Sex Bias and Omission in Neuroscience Research