The Cancer Genomics Hub (CGHub): overcoming cancer through the power of torrential data

Christopher Wilks,Melissa S. Cline,Erich Weiler,Mark Diehkans,Brian Craft,Christy Martin,Daniel Murphy,Howdy Pierce,John Black,Donavan Nelson,Brian Litzinger,Thomas Hatton,Lori Maltbie,Michael Ainsworth,Patrick Allen,Linda Rosewood,Elizabeth Mitchell,Bradley Smith,Jim Warner,John Groboske,Haifang Telc,Daniel Wilson,Brian Sanford,Hannes Schmidt,David Haussler,Daniel Maltbie,C. Wilks,M. S. Cline,E. Weiler,M. Diehkans,B. Craft,C. Martin,D. Murphy,H. Pierce,J. Black,D. Nelson,B. Litzinger,T. Hatton,L. Maltbie,M. Ainsworth,P. Allen,L. Rosewood,E. Mitchell,B. Smith,J. Warner,J. Groboske,H. Telc,D. Wilson,B. Sanford,H. Schmidt,D. Haussler,D. Maltbie
DOI: https://doi.org/10.1093/database/bau093
2014-09-29
Database
Abstract:The Cancer Genomics Hub (CGHub) is the online repository of the sequencing programs of the National Cancer Institute (NCI), including The Cancer Genomics Atlas (TCGA), the Cancer Cell Line Encyclopedia (CCLE) and the Therapeutically Applicable Research to Generate Effective Treatments (TARGET) projects, with data from 25 different types of cancer. The CGHub currently contains >1.4 PB of data, has grown at an average rate of 50 TB a month and serves >100 TB per week. The architecture of CGHub is designed to support bulk searching and downloading through a Web-accessible application programming interface, enforce patient genome confidentiality in data storage and transmission and optimize for efficiency in access and transfer. In this article, we describe the design of these three components, present performance results for our transfer protocol, GeneTorrent, and finally report on the growth of the system in terms of data stored and transferred, including estimated limits on the current architecture. Our experienced-based estimates suggest that centralizing storage and computational resources is more efficient than wide distribution across many satellite labs. Database URL: https://cghub.ucsc.edu.
What problem does this paper attempt to address?