Evaluating NoSQL Databases for Big Data Processing within the Brazilian Ministry of Planning, Budget, and Management

Ruben C. Huacarpuma,Daniel da C. Rodrigues,Antonio M. Rubio Serrano,João Paulo C. Lustosa da Costa,Rafael T. de Sousa Júnior,Lizane Leite,Edward Ribeiro,Maristela Holanda,Aleteia P. F. Araujo
DOI: https://doi.org/10.4018/978-1-4666-9840-6.ch050
IF: 4.426
2016-01-01
Big Data
Abstract:The Brazilian Ministry of Planning, Budget, and Management (MP) manages enormous amounts of data that is generated on a daily basis. Processing all of this data more efficiently can reduce operating costs, thereby making better use of public resources. In this chapter, the authors construct a Big Data framework to deal with data loading and querying problems in distributed data processing. They evaluate the proposed Big Data processes by comparing them with the current centralized process used by MP in its Integrated System for Human Resources Management (in Portuguese: Sistema Integrado de Administração de Pessoal – SIAPE). This study focuses primarily on a NoSQL solution using HBase and Cassandra, which is compared to the relational PostgreSQL implementation used as a baseline. The inclusion of Big Data technologies in the proposed solution noticeably increases the performance of loading and querying time.
computer science, theory & methods, interdisciplinary applications
What problem does this paper attempt to address?