Abstract:Data Integration is becoming very important in many commercial applications and scientific research. A lot of algorithms and systems have been proposed and developed to address related issues from different aspects. Virtual database systems are well-recognized as one of the effective solutions of data integration. The existing execution modules in virtual database systems are very ineffective. MapReduce (MR) is a new computing model for parallel processing and has a good performance on large-scale data execution. In this paper, we propose a new distributed data integration system, called VDB-MR, which is based on the MapReduce technology, to efficiently integrate data from heterogeneous data sources. With VDB-MR, a unified view (i.e., a single virtual database) of multiple databases can be provided to users. We also conducted a series of experiments to evaluate VDB-MR by comparing it with an open source data integration system OGSA-DAI and two DBMSs in parallel. Experiment results show that VDB-MR significantly outperforms OGSA-DAI and the DBMSs in parallel.

Study on virtual database architecture of data integration

IMPLEMENT DATA SHARING OVER NETWORK HETEROGENEOUS DATABASES BY DATA DIMENSION REDUCTION METHOD

Metadata Management Technology in Data Warehouse

Research of Heterogeneous Database Integration Platform Based on Virtual View

Research on Information Integration Platform for Gas Field Based on Virtual Data Center

VDM: Virtual Database Management for Distributed Databases and File Systems

Integrated System of Heterogeneous Data Based on Web Services

Research of Distributed Heterogeneous Data integration Based on Metadata

Research on Virtual Construction Information Integration Management

Virtual Database And Decision Support

VDB-MR: MapReduce-based distributed data integration using virtual database

Multi-Source Heterogeneous Data Integration Technology and Its Development

Research on Virtual Data Center Architecture Based on Intelligent Multi-agent

Design of Digital Library Information Retrieval System

Research and Application of Data Modeling and Integration Based on Metadata

Research on 4 G Geological Spatial Data Integration Technology Based on Data Warehouse

Detection and Resolution of Data Confliction in the Integration of Heterogeneous Information Sources

An Integration of Heterogeneous Resources Based on Semantic Grid

Integration of Medical Information Systems Based on Virtual Database and Web Services

Research of cloud computing based agriculture virtualized information database

Semantic Based Integration Of Heterogeneous Databases