Fundamentals of big data in radio astronomy

Jiale Lei,Linghe Kong
DOI: https://doi.org/10.1016/b978-0-12-819084-5.00010-9
2020-01-01
Abstract:Abstract Large digital sky surveys are becoming the dominant source of data in astronomy. There are more than 100 terabytes of data in major archives, and that amount is growing rapidly. A typical sky survey archive has approximately 10 terabytes of image data and a billion detected sources (stars, galaxies, quasars, etc.), with hundreds of measured attributes per source. These surveys span the full range of wavelengths, radio through gamma ray, yet they are just a taste of the much larger datasets to come. Yearly advances in electronics bring new instruments that double the amount of data collected each year and lead to the exponential growth of information in astronomy. Thus, datasets that are orders of magnitude larger, more complex, and more homogeneous than in the past are on the horizon. In comparison, the size of the human genome is about 1 gigabyte and that of the Library of Congress is about 20 terabytes. Truly, astronomy has come to the big data era.
What problem does this paper attempt to address?