Abstract:The Internet of Things (IoT) has recently received considerable interest due to the development of smart technologies in today's interconnected world. With the rapid advancement in Internet technologies and the proliferation of IoT sensors, myriad systems and applications generate data of a massive volume, variety and velocity which traditional databases and systems are unable to manage effectively. Many organizations need to deal with these massive datasets that encounter different types of data (e.g., IoT streaming data, static data) in different formats (e.g., structured, semi-structured) coming from multiple sources. Several data integration mechanisms have been designed to process mostly static data. Unfortunately, these techniques are not able to deal with and integrate IoT streaming datasets from multiple sources. In this paper, we identify the challenges of IoT Streaming Data Integration (ISDI) and present a formal approach for the real-time integration of such IoT streaming datasets. We address one of the important issues of timing conflict/alignment among streaming data coming from multiple sources. A generic window-based ISDI approach is proposed to deal with IoT data in different formats and algorithms are developed to integrate IoT streaming data from multiple sources. In particular, we extend the basic windowing algorithm for real-time data integration and to deal with the timing alignment issue. We also introduce a de-duplication algorithm to deal with data redundancy and to demonstrate the useful fragments of the integrated data. We conduct several sets of experiments and quantify the performance of our proposed window-based approach. In particular, we compare our local experimental results with a real setup for streaming data, using Apache Spark. The results of the experiments, which are performed on several IoT datasets, show the efficiency of our proposed solution in terms of processing time. The results are also used to provide an integrated data view to the users.

A Heterogeneous Data Integration Model

An Application-Driven Heterogeneous Internet Of Things Integration Architecture

Research on data mining models for the internet of things

Design and Implementation of Mediation-based Heterogeneous Data Integration HDIS System

Integrated System of Heterogeneous Data Based on Web Services

Hetrogeneous information integrations from internet of things based on PML and Hedge automata

Heterogeneous Data Integration and Fusion System Based on Metadata Conflict Algorithms in USPIOT

IoT Lakehouse: A New Data Management Paradigm for AIoT.

Data Ming Model Based on Cloud Computing in the Internet of Things

Dynamic Description Logic Model for Data Integration.

Grid-based Integration Model of Distributed and Heterogeneous Data

An IoT Middleware of Data Service

A Flexible Data Structure for Heterogeneous Information Integration

Data and Decision Intelligence for Internet of Things: Putting Human in the Loop

IoT streaming data integration from multiple sources

Heterogeneous Application Domains Model Driven Resource Integration

A universal heterogeneous data integration standard and parse algorithm in real-time database

Human-in-the-loop Data Integration

Heterogeneous data format integration and conversion (HDFIC) using machine learning and IBM-DFDL for IoT

Energy Analysis and Application of Data Mining Algorithms for Internet of Things Based on Hadoop Cloud Platform

A Deep Residual Computation Model for Heterogeneous Data Learning in Smart Internet of Things