An Adaptive Elastic Multi-model Big Data Analysis and Information Extraction System
Qiang Yin,Jianhua Wang,Sheng Du,Jianquan Leng,Jintao Li,Yinhao Hong,Feng Zhang,Yunpeng Chai,Xiao Zhang,Xiaonan Zhao,Mengyu Li,Song Xiao,Wei Lu
DOI: https://doi.org/10.1007/s41019-022-00196-2
2022-01-01
Data Science and Engineering
Abstract:With the diverse applications to industry and domain-specific context, multi-source information extraction on semi-structured and unstructured data, as well as across data models, is becoming more common. However, multi-model information extraction often requires the deployment of multiple data model management, storage, and analysis subsystems on the cloud, many subsystems are not high-resource utilization at the same time, and the resource waste phenomenon is often serious. Therefore, an adaptive scalable multi-model big data analysis and information extraction system is designed and implemented in this paper, which can support data maintenance and cross-model query of relational, graph, document, key and other data models, and can provide efficient cross-model information extraction. On this basis, we can achieve the system resource allocation on demand and fast scaling mechanism, according to the real-time requirements of multi-model big data analysis, and dynamic adjustment of each subsystem resource allocation. Therefore, our solution not only guarantees multi-model query and information extraction performance and quality of service, but also significantly reduces the total consumption of system resources and cost.