Reproducibility of in-vivo electrophysiological measurements in mice
The International Brain Laboratory,Kush Banga,Julius Benson,Jai Bhagat,Dan Biderman,Daniel Birman,Niccolò Bonacchi,Sebastian A Bruijns,Kelly Buchanan,Robert A Campbell,Matteo Carandini,Gaëlle A Chapuis,Anne K Churchland,M Felicia Davatolhagh,Hyun Dong Lee,Mayo Faulkner,Berk Gerçek,Fei Hu,Julia M Huntenburg,Cole Hurwitz,Anup Khanal,Christopher Krasniak,Christopher Langfield,Guido T Meijer,Nathaniel J Miska,Zeinab Mohammadi,Jean-Paul Noel,Liam Paninski,Alejandro Pan-Vazquez,Noam Roth,Michael Schartner,Karolina Socha,Nicholas A Steinmetz,Karel Svoboda,Marsa Taheri,Anne E Urai,Miles Wells,Steven J West,Matthew R Whiteway,Olivier Winter,Ilana B Witten
DOI: https://doi.org/10.1101/2022.05.09.491042
2024-06-20
Abstract:Understanding brain function relies on the collective work of many labs generating reproducible results. However, reproducibility has not been systematically assessed within the context of electrophysiological recordings during cognitive behaviors. To address this, we formed a multi-lab collaboration using a shared, open-source behavioral task and experimental apparatus. Experimenters in ten laboratories repeatedly targeted Neuropixels probes to the same location (spanning secondary visual areas, hippocampus, and thalamus) in mice making decisions; this generated a total of 121 experimental replicates, a unique dataset for evaluating reproducibility of electrophysiology experiments. Despite standardizing both behavioral and electrophysiological procedures, some experimental outcomes were highly variable. A closer analysis uncovered that variability in electrode targeting hindered reproducibility, as did the limited statistical power of some routinely used electrophysiological analyses, such as single-neuron tests of modulation by task parameters. Reproducibility was enhanced by histological and electrophysiological quality-control criteria. Our observations suggest that data from systems neuroscience is vulnerable to a lack of reproducibility, but that across-lab standardization, including metrics we propose, can serve to mitigate this.
Neuroscience