Making Big Data Intelligent Storable at the Edge: Storage Resource Intelligent Orchestration

Fuli Qiao,Mianxiong Dong,Kaoru Ota,Siyi Liao,Jun Wu,Jianhua Li

DOI: https://doi.org/10.1109/globecom38437.2019.9013942

2019-01-01

Abstract:Network edge equipment has generated a large amount of fast- growing data, which has placed a heavy burden on the collaboration of heterogeneous networks. Due to the diversity of edge computing application scenarios, many new requirements are advocated for unified data storage management, such as latency and processing efficiency. Traditional centralized cloud storage can no longer meet the on- demand of edge computing in the case of a surge in data volume. Therefore, a unified storage architecture is required for the current improvements in computational offloading schemes and storage optimization algorithms. To solve these challenges and make data intelligent collaborative storable, this paper proposes a novel unified storage architecture for big data in the edge-cloud, which supports edge services in order to extend Hadoop at the edge. The functions of the edge nodes are proposed to synchronize the edge nodes of the same neighborhood and store data dynamically via Q- learning based on popularity, in order to mitigate network load pressure and improve the efficiency of edge services. An intelligent scheme that impacts the quality of service (QoS) through data marginal storage is proposed to improve the resource scheduling and to the distribution of storage space. Simulation results demonstrate the merits and efficiency of the proposed intelligent architecture is superior to the comparison schemes.