Firework: Big Data Sharing and Processing in Collaborative Edge Environment
Quan Zhang,Xiaohong Zhang,Qingyang Zhang,Weisong Shi,Hong Zhong
DOI: https://doi.org/10.1109/HotWeb.2016.12
2016-01-01
Abstract:Cloud computing, arguably, has become the de facto computing platform for the big data processing by researchers and practitioners for the last decade, and enabled different stakeholders to discover valuable information from large scale data. At the same time, in the decade, we have witnessed the fast growing deployment of billions of sensors and actuators in multiple applications domains, such as transportation, manufacturing, connected/wearable health care, smart city and so on, stimulating the emerging of Edge Computing (a.k.a., fog computing, cloudlet). However, data, as the core of both cloud computing and edge computing, is still owned by each stakeholder and rarely shared due to privacy concern and formidable cost of data transportation, which significantly limits Internet of Things (IoT) applications that need data input from multiple stakeholders (e.g., video analytics collects data from cameras owned by police department, transportation department, retailer stores, etc.). In this paper, we envision that in the era of IoT the demand of distributed big data sharing and processing applications will dramatically increase since the data producing and consuming are pushed to the edge of the network. Data processing in collaborative edge environment needs to fuse data owned by multiple stakeholders, while keeping the computation within stakeholders' data facilities. To attack this challenge, we propose a new computing paradigm, Firework, which is designed for big data processing in collaborative edge environment (CEE). Firework fuses geographically distributed data by creating virtual shared data views that are exposed to end users via predefined interfaces by data owners. The interfaces are provided in the form of a set of datasets and a set of functions, where the functions are privacy preserved and bound to the datasets. Firework targets to share data while ensuring data privacy and integrity for stakeholders. By pushing the data processing as close as to data sources, Firework also aims to avoid data movement from the edge of the network to the cloud and improve the response latency.