Benchmarking of T-Cell Receptor - Epitope Predictors with ePytope-TCR

Felix Drost,Anna Chernysheva,Mahmoud Albahah,Katharina Kocher,Kilian Schober,Benjamin Schubert
DOI: https://doi.org/10.1101/2024.11.06.622261
2024-11-08
Abstract:Understanding the recognition of disease-derived epitopes through T-cell receptors (TCRs) has the potential to serve as a stepping stone for the development of efficient immunotherapies and vaccines. While a plethora of sequence-based prediction methods for TCR-epitope binding exists, their available pre-trained models have not been comparatively evaluated on standardized datasets and evaluation settings. Furthermore, technical problems such as non-standardized input and output formats of these prediction tools hinder interoperability and broad usage in applied research. To alleviate these shortcomings, we introduce ePytope-TCR, an extension of the vaccine design and immuno-prediction framework ePytope. We integrated 18 TCR-epitope prediction methods into this common framework offering interoperable interfaces with standard TCR repertoire data formats. We showcase the applicability of ePytope-TCR by evaluating the performance of the prediction methods on two challenging datasets for annotating single-cell repertoires and predicting TCR cross-reactivity towards mutated epitopes. While novel predictors successfully predicted binding to frequently observed epitopes, all methods failed for less observed epitopes. Further, we detected a strong bias in the prediction scores between different epitope classes. We envision this benchmark to guide researchers in their choice of a predictor for a given setting. Further, we aspire to accelerate the development of novel prediction models by allowing fast benchmarking against existing approaches through common interfaces and defining standardized evaluation settings.
Bioinformatics
What problem does this paper attempt to address?