A solution of data inconsistencies in data integration

WangXin,HuangLin-Peng,ZhangYi,XuXiao-Hui,ChenJun-Qing

IF: 1.871

2010-01-01

Journal of Computer Science and Technology

Abstract:New challenges including how to share information on heterogeneous devices appear in data-intensive pervasive computing environments. Data integration is a practical approach to these applications....

What problem does this paper attempt to address?

A Solution of Data Inconsistencies in Data Integration — Designed for Pervasive Computing Environment

Xin Wang,Lin-Peng Huang,Yi Zhang,Xiao-Hui Xu,Jun-Qing Chen

DOI: https://doi.org/10.1007/s11390-010-9340-2

IF: 1.871

2010-01-01

Journal of Computer Science and Technology

Abstract:New challenges including how to share information on heterogeneous devices appear in data-intensive pervasive computing environments. Data integration is a practical approach to these applications. Dealing with inconsistencies is one of the important problems in data integration. In this paper we motivate the problem of data inconsistency solution for data integration in pervasive environments. We define data quality criteria and expense quality criteria for data sources to solve data inconsistency. In our solution, firstly, data sources needing high expense to obtain data from them are discarded by using expense quality criteria and utility function. Since it is difficult to obtain the actual quality of data sources in pervasive computing environment, we introduce fuzzy multi-attribute group decision making approach to selecting the appropriate data sources. The experimental results show that our solution has ideal effectiveness.
A Solution for Data Inconsistency in Data Integration

Xin Wang,Lin-Peng Huang,Xiao-Hui Xu,Yi Zhang,Jun-Qing Chen

2011-01-01

Journal of information science and engineering

Abstract:Data integration is a problem of combining data residing at different sources and providing the user with a unified view of these data. An important issue in data integration is the possibility of conflicts among the different data sources. Data sources may conflict with each other at data value level which is defined as data inconsistency. So in this paper, a solution for data inconsistency in data integration is proposed. An approximate object-oriented data model extended with data source quality criteria is defined. On the basis of our data model, we provide a data inconsistency solution strategy. To accomplish our strategy, fuzzy multi-attribute decision making approach based on data source quality criteria is applied to select the "best" data source's data as the data inconsistency solution. A set of experiments is designed and performed to evaluate the effectiveness of our strategy and algorithm. The experimental results indicate that our solution performs ideally.
IMPLEMENT DATA SHARING OVER NETWORK HETEROGENEOUS DATABASES BY DATA DIMENSION REDUCTION METHOD

Hua JIANG,Qi CHEN,Rui-Zhao YU

2000-01-01

Journal of Computer Research and Development

Abstract:The variety of databases structures as well as multiple data correlation on database index sets are major barriers on data sharing between heterogeneous databases. Most research work currently centralized on coherence access methods over heterogeneous databases, however, those methods can't establish a system in good scalability. In this paper, a new resolution is presented, which firstly find out general characteristics between many heterogeneous databases, and then the complexity of each database is reduced by using dimension reduction methods, so that data\|sharing becomes more flexible with a good scalability. In addition, the principal component views(PCV) reduce data correlation between shared\|databases.
Practicability of Dataspace Systems

Hamid Turab Mirza,Ling Chen,Gencai Chen

DOI: https://doi.org/10.4156/jdcta.vol4.issue3.23

2010-01-01

International Journal of Digital Content Technology and its Applications

Abstract:Nowadays there is rarely a scenario where all the data can be fit nicely into one relational database management system or any other single data model. In acknowledgement of this fact a new concept of Dataspaces was introduced, according to which a dataspace system is assumed to be a hybrid of a search engine, a databases management system, an information integration system and a data sharing system. However the concept was presented in a visionary way and its implementation in the real world scenario has opened up many complex research challenges. Moreover so far the efforts put forth by the research community are quite disparate and it is highly desirable to have a unified effort which would hopefully enable rapid progress. Furthermore due to very high end-user expectations of such systems there are a lot of challenges and problems that need to be resolved and much scope for future work remains.
Towards Merging Pervasive Computing into Grid - Lightweight Portal, Dynamic Collaborating and Semantic Supporting

Wei Shi,Shanping Li,Xin Lin

DOI: https://doi.org/10.1109/imsccs.2006.137

2006-01-01

Abstract:The emerging grid infrastructure is one of the key perspectives of pervasive computing. However, it is difficult to implement all the grid semantics on pervasive devices due to the inherent limitations of most pervasive devices. It is not yet clear that how to organize large quantities of heterogeneous pervasive devices collaborating with each other to solve user-centered tasks. And exploiting grid infrastructure to accomplish resource-intensive tasks is still a challenge for resource-weak pervasive devices. In this paper, we put forward the concept of user's task oriented nomadic autonomy domain (UTONAD) which models the pervasive devices collaborating patterns when they are included in a global resource-sharing platform. Based on UTONAD, we propose the architecture of our implementation that provides a uniform method to manage QoS guaranteed interactions between pervasive devices and grid in a dynamic (both temporal and spatial) and user-transparent fashion
A Review of Data Integration

Chen Yue

2004-01-01

Abstract:Firstly, the concepts and difficulties of data integration are introduced. Secondly, three typical data integration approaches: Schema Integration, Data Replication and Hybrid Approach are discussed and compared-Then, heterogeneity problem, a main difficulty of data integration, is explained in detail. Finally, further research fields are suggested.
Solution to Some Problems in Data Integration by Using XML

DENG Guo-jun,FU Yan

DOI: https://doi.org/10.3969/j.issn.1671-1742.2006.04.009

2006-01-01

Abstract:The condition of the database system is analyzed and the concept of the information island is presented.The data integration is an effective method to solve the information island.The problems such as different structure source database,conflict in means and visual control in the front of user are analyzed in detail.A method to solve the problems in the data integration by using configuration file of XML as middle model is presented.Thus the data integration can effectively solve the problem of the information island and make the job of data integration more easily.
Data Integration Method and Application under Distributed Heterogeneous Environment

CHEN Hong,CAO Jian,LIANG Min

DOI: https://doi.org/10.3969/j.issn.1000-3428.2005.05.039

2005-01-01

Abstract:The paper presents a data integration and exchange framework under distributed heterogeneous environment based on XML and Web services. The detailed functions of the framework are described. The methods are totally implemented in the data exchange platform of Wuxi BTV for eliminating isolated information islands.
To Realize Data Exchange Among Multiple Systems

Yongqin Yang,Zhijun He

DOI: https://doi.org/10.3321/j.issn:1002-8331.2001.02.023

2001-01-01

Abstract:A common method is presented for data exchange among the existing application systems of enterprises.With the method,the problem of inconsistency of table data structure among systems is effectively solved,and which data tables can be exchanged is also not limited.
Research on Key Problems in Data Integration System

LIU Min-chao,LIU Wei-dong

2006-01-01

Abstract:The notion of data integration was introduced and the aim of building data integration systems was presented.Some key problems in building data integration system were explained.They were building semantic relation,query rewriting,query optimizing and executing.Current researches on these problems were described in detail,and some thoughts for resolving them were proposed.
Data Fusion: Resolving Conflicts from Multiple Sources

Xin Luna Dong,Laure Berti-Equille,Divesh Srivastava

DOI: https://doi.org/10.48550/arXiv.1503.00310

2015-03-02

Abstract:Many data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, require integrating data from multiple sources. Each of these sources provides a set of values and different sources can often provide conflicting values. To present quality data to users, it is critical to resolve conflicts and discover values that reflect the real world; this task is called {\em data fusion}. This paper describes a novel approach that finds true values from conflicting information when there are a large number of sources, among which some may copy from others. We present a case study on real-world data showing that the described algorithm can significantly improve accuracy of truth discovery and is scalable when there are a large number of data sources.

Databases
A Grid Middleware DISQ for Data Integration

Xin Wang,Linpeng Huang,Yi Zhang

DOI: https://doi.org/10.1109/csse.2008.699

2008-01-01

Abstract:The Grid offers new opportunities and raises new challenges for data integration. The Grid communities have paid great attention to data integration middle ware in grid environment. But few of those projects actually establish semantic connections among heterogeneous data sources. Recently, several resolutions of data integration in grid environment have been proposed. However most of them put their emphasis in a particular data integration domain such as medical data integration than a common grid data integration middle ware. So in this paper we provide a grid middle ware DISQ for semantic data integration based on a data model which is extended for data inconsistency solution to improve the quality of data integration by introducing data source quality criteria into the data model. It provides application interfaces to user and to semantically integrate data sources based on our data model, it encapsulates services provided by OGSA-DAI which only shields data sources' physical access heterogeneity.
Research of Matching Technology in Data Integration

XU Feng,MAN Zhenmei,WANG Zhijian

DOI: https://doi.org/10.3969/j.issn.1000-3428.2006.06.014

2006-01-01

Abstract:With the development of network technology,it is necessary for information processing to adopt an unity accessing method and form for vast networks of large scale,heterogeneous and distributed computing resource,which is data integration’s work.Schema matching is a basic problem in the field of data integration;the main aspect of it concerns a match between global schema and local schema.This paper proposes solutions to solve these conflicts and integrate the schemas into a unified one.
Multi-Source Spatial Data Integration: Problems and Some Solutions

M. Wang,P.J. Howarth

DOI: https://doi.org/10.1080/07038992.1994.10874579

IF: 2.6

1994-12-01

Canadian Journal of Remote Sensing

Abstract:SUMMARYIt is recognized that spatial data from different sources are not always initially compatible because of differences in the way the data are acquired. To be able to integrate different types of spatial data satisfactorily, it is important to look at the problem from both conceptual and practical viewpoints. From a geographer's viewpoint, spatial objects stored in a spatial data base are abstract models of the real world. Because of the way in which spatial data are generalized — specifically the generalization of spatial objects — spatial data from different sources can vary in terms of the data structures within the data base (vector/raster), the data types involved (nominal/numerical), the spatial resolutions (size of spatial units) and geometrical characteristics, and the levels of generalization (detail of information). An example is the differences between a thematic map and an image. These differences can make it difficult to combine spatial data sets for further processing. In this paper, difficulties in combining spatial data sets are investigated in terms of their incompatibilities. Two basic issues in spatial data integration — spatial data base compilation and spatial data analysis — are discussed. Four practical approaches are proposed to handle the incompatibilities between spatial data sets.

remote sensing
Heterogeneous data integration: Challenges and opportunities

I Made Putrama,Péter Martinek

DOI: https://doi.org/10.1016/j.dib.2024.110853

2024-08-29

Abstract:Integrating multiple data source technologies is essential for organizations to respond to highly dynamic market needs. Although physical data integration systems have been considered to have better query processing systems, they pose higher implementation and maintenance costs. Meanwhile, virtual data integration has become an alternative topic that is increasingly attracting the attention of researchers in the current era of big data. Various data integration methodologies have been developed and used in various domains, processing heterogeneous data using various approaches. This review article aims to provide an overview of heterogeneous data integration research focusing on methodology and approaches. It surveys existing publications, highlighting key trends, challenges, and open research topics. The main findings are: (i) Research has been conducted in various domains. However, most focus on big data rather than specific study domains; (ii) researchers primarily focus on semantics challenges, and (iii) gaps still need to be addressed and related to integration issues involving semantics and unstructured data formats that must be thoroughly addressed. Furthermore, considering elements of cutting-edge technology, such as machine learning and data integration, about privacy concerns provides a chance for additional investigation. Finally, we provide insight into the potential for a broader review of integration challenges based on case studies.
An Integration Scheme For Data Sharing Under University Resource Planning

Yongkai Fan,Dongxing Jiang,Xiaolong Fu,Tianze Sun

2008-01-01

Abstract:Interoperation of data sources which differ structurally and semantically could result in information inconsistent, and coexistence of heterogeneous databases had its historical reasons in universities information systems. Timely access to the newest and consistent resources is needed. In this paper, the design of data integration framework is proposed to tackle the inconsistent information problem under the direction of University Resource Planning theory. There are two important components for the integration of heterogeneous systems: mapping and integration toolkit. With mapping, original data sources can be unified according to the business regions divided. Integration toolkit includes wrappers, mediator and other transformation tool for materialization data warehouse.
Toward Spreadsheet-Based Data Management in Distributed Enterprise Environment

JF Tang,B Zhou,ZJ He,P Uros

DOI: https://doi.org/10.1109/cacwd.2004.1349257

2004-01-01

Abstract:We propose a feasible approach on managing the heterogeneous data in distributed enterprise environment, which leverages the born data presentation and calculation ability of spreadsheet to unify the enterprise information access to heterogeneous data sources through its core component: universal distributed data interchanging middleware (UDDIM). Afterwards, we focus on the problem of unifying all kinds of heterogeneous data sources from heterogeneous OS in Intranet through CORBA and Internet through SOAP into one single "Spreadsheet World". And this "Spreadsheet World" is bridged to and fro other information worlds through information gateway technology. XML technique is used to achieve data exchanging regardless of the differences of low-level communication mechanisms. As a result, all clients can access and utilize information from one single "Spreadsheet World" in variety of ways, both powerfully and easily.
XML-based Complex Lab Data Representing, Storing and Publishing

Shao Zhijiang,Wang Yongming

DOI: https://doi.org/10.1109/wcica.2004.1342196

2004-01-01

Abstract:The problem of heterogeneous data integration in industrial supervisory control is presented and analyzed. The demand and difficulties of complex lab data representing, storing and publishing are outlined and analyzed in this paper. It is pointed out that a self-descriptive data structure with clear organization of contents and easy connection to human-machine interface is needed. A solution based on XML (eXtended Mark-up Language) is proposed with combination of different layers of data storage, data logic and presentation, human-machine interface. An XML-based system is developed to meet the demands of laboratory data publishing with complex structure. Construction of user interface from XML schema, conversion of lab data into XML, algorithm of storing XML into RDBMS (relation database manager system), query and reconstruction of XML documents from the RDBMS are touched upon. The data structure is described in details and its unique advantage is presented.
An intermediate view for data integration, management in cloud computing

Jingjun Ge,Changjun Hu,Yang Li,Xin Liu,Haolei Zuo

DOI: https://doi.org/10.12733/jcis5937

2013-01-01

Journal of Computational Information Systems

Abstract:It is necessary to integrate and manage data in the Cloud for providing in-depth data services. This paper introduces an intermediate view that provides a highly interactive environment to integrate and manage data distributed in different sources. The intermediate view, emphasizing on the notion of payas-you-go data integration and management, provides Data sources Integration Broker, Data Sources Intermediate View Model and Data Sources View Manager that can automatically create mappings between data sources to support integrating, viewing and querying data sources relationship, integrates multiple search/query strategies, and provides view query operations for users to manage data more easily. Finally, we report on successful application results obtained with a prototype implementation called MSDSN, involving challenging management and sharing of materials scientific data, to illustrate the feasibility and effectiveness of intermediate view. © 2013 by Binary Information Press.
The Research On Customer Data Integration Technology Based On Web Services

Js Su,Xj Gu,Zg Bao

2005-01-01

Abstract:Along with the popularization of the automobiles, customer management and supply chain management of automobile parts is becoming more and more important. Customer data is an important resource of automobile parts company,it should be shared and integrated for the forecast of customer and company decision-making. The problem of distributed data transmission can be solved by using XML and Web services. It provides a cooperative tool for different application system in different places. J2EE can be used to solve the problem of data integration. In this paper, the transmission of customer data through network is investigated, the method for data collection is analyzed, the architecture system of the data integration is designed and analyzed, and the data collection is investigated.

A solution of data inconsistencies in data integration

A Solution of Data Inconsistencies in Data Integration — Designed for Pervasive Computing Environment

A Solution for Data Inconsistency in Data Integration

IMPLEMENT DATA SHARING OVER NETWORK HETEROGENEOUS DATABASES BY DATA DIMENSION REDUCTION METHOD

Practicability of Dataspace Systems

Towards Merging Pervasive Computing into Grid - Lightweight Portal, Dynamic Collaborating and Semantic Supporting

A Review of Data Integration

Solution to Some Problems in Data Integration by Using XML

Data Integration Method and Application under Distributed Heterogeneous Environment

To Realize Data Exchange Among Multiple Systems

Research on Key Problems in Data Integration System

Data Fusion: Resolving Conflicts from Multiple Sources

A Grid Middleware DISQ for Data Integration

Research of Matching Technology in Data Integration

Multi-Source Spatial Data Integration: Problems and Some Solutions

Heterogeneous data integration: Challenges and opportunities

An Integration Scheme For Data Sharing Under University Resource Planning

Toward Spreadsheet-Based Data Management in Distributed Enterprise Environment

XML-based Complex Lab Data Representing, Storing and Publishing

An intermediate view for data integration, management in cloud computing

The Research On Customer Data Integration Technology Based On Web Services