Highland games: A benchmarking exercise in predicting biophysical and drug properties of monoclonal antibodies from amino acid sequences
Jonathan Coffman,Bruno Marques,Raquel Orozco,Minni Aswath,Hasan Mohammad,Eike Zimmermann,Joelle Khouri,Jan Griesbach,Saeed Izadi,Ambrose Williams,Kannan Sankar,Benjamin Walters,Jasper Lin,Stefan Hepbildikler,John Schiel,John Welsh,Gisela Ferreira,Jared Delmar,Neil Mody,Christopher Afdahl,Tingting Cui,Rushd Khalaf,Alexander Hanke,Lars Pampel,Siddharth Parimal,Xuan Hong,Ujwal Patil,Jennifer Pollard,Francis Insaidoo,Julie Robinson,Divya Chandra,Marco Blanco,Jainik Panchal,Soundara Soundararajan,David Roush,Nihal Tugcu,Steven Cramer,Charles Haynes,Richard C Willson
DOI: https://doi.org/10.1002/bit.27349
Abstract:Biopharmaceutical product and process development do not yet take advantage of predictive computational modeling to nearly the degree seen in industries based on smaller molecules. To assess and advance progress in this area, spirited coopetition (mutually beneficial collaboration between competitors) was successfully used to motivate industrial scientists to develop, share, and compare data and methods which would normally have remained confidential. The first "Highland Games" competition was held in conjunction with the October 2018 Recovery of Biological Products Conference in Ashville, NC, with the goal of benchmarking and assessment of the ability to predict development-related properties of six antibodies from their amino acid sequences alone. Predictions included purification-influencing properties such as isoelectric point and protein A elution pH, and biophysical properties such as stability and viscosity at very high concentrations. Essential contributions were made by a large variety of individuals, including companies which consented to provide antibody amino acid sequences and test materials, volunteers who undertook the preparation and experimental characterization of these materials, and prediction teams who attempted to predict antibody properties from sequence alone. Best practices were identified and shared, and areas in which the community excels at making predictions were identified, as well as areas presenting opportunities for considerable improvement. Predictions of isoelectric point and protein A elution pH were especially good with all-prediction average errors of 0.2 and 1.6 pH unit, respectively, while predictions of some other properties were notably less good. This manuscript presents the events, methods, and results of the competition, and can serve as a tutorial and as a reference for in-house benchmarking by others. Organizations vary in their policies concerning disclosure of methods, but most managements were very cooperative with the Highland Games exercise, and considerable insight into common and best practices is available from the contributed methods. The accumulated data set will serve as a benchmarking tool for further development of in silico prediction tools.