Abstract:As deep learning, virtual reality, and other technologies become mature, real-time data processing applications running on intelligent terminals are emerging endlessly; meanwhile, edge computing has developed rapidly and has become a popular research direction in the field of distributed computing. Edge computing network is a network computing environment composed of multi-edge computing nodes and data centers. First, the edge computing framework and key technologies are analyzed to improve the performance of real-time data processing applications. In the system scenario where the collaborative deployment tasks of multi-edge nodes and data centers are considered, the stream processing task deployment process is formally described, and an efficient multi-edge node-computing center collaborative task deployment algorithm is proposed, which solves the problem of copy-free task deployment in the task deployment problem. Furthermore, a heterogeneous edge collaborative storage mechanism with tight coupling of computing and data is proposed, which solves the contradiction between the limited computing and storage capabilities of data and intelligent terminals, thereby improving the performance of data processing applications. Here, a Feasible Solution (FS) algorithm is designed to solve the problem of placing copy-free data processing tasks in the system. The FS algorithm has excellent results once considering the overall coordination. Under light load, the V value is reduced by 73% compared to the Only Data Center-available (ODC) algorithm and 41% compared to the Hash algorithm. Under heavy load, the V value is reduced by 66% compared to the ODC algorithm and 35% compared to the Hash algorithm. The algorithm has achieved good results after considering the overall coordination and cooperation and can more effectively use the bandwidth of edge nodes to transmit and process data stream, so that more tasks can be deployed in edge computing nodes, thereby saving time for data transmission to the data centers. The end-to-end collaborative real-time data processing task scheduling mechanism proposed here can effectively avoid the disadvantages of long waiting times and unable to obtain the required data, which significantly improves the success rate of the task and thus ensures the performance of real-time data processing.

High-throughput Real-time Edge Stream Processing with Topology-Aware Resource Matching

Operator Placement for IoT Data Streaming Applications in Edge Computing Environment.

An Intelligent Resource Scheduling Method with Edge Channel Deployment for BPM.

TATA: Throughput-Aware TAsk Placement in Heterogeneous Stream Processing with Deep Reinforcement Learning

Edge-Stream: a Stream Processing Approach for Distributed Applications on a Hierarchical Edge-computing System

Amnis: Optimized stream processing for edge computing

Resource- and Message Size-Aware Scheduling of Stream Processing at the Edge with application to Realtime Microscopy

AI-enabled IoT-Edge Data Analytics for Connected Living

Topology-Aware Task Allocation For Distributed Stream Processing With Latency Guarantee

AgileDART: An Agile and Scalable Edge Stream Processing Engine

An On-the-Fly Scheduling Strategy for Distributed Stream Processing Platform.

Mobile Resource Aware Scheduling for Mobile Edge Environment

Two-stage Scheduling of Stream Computing for Industrial Cloud-edge Collaboration

Topology-aware task allocation for online distributed stream processing applications with latency constraints

RIoTBench: A Real-time IoT Benchmark for Distributed Stream Processing Platforms

Runtime-Aware Adaptive Scheduling in Stream Processing

Cost-Effective Task Scheduling for Collaborative Cross-Edge Analytics

Hybrid Workflow Scheduling on Edge Cloud Computing Systems

Adaptive Topology Decomposition for Storm

Adaptive scheduling of continuous operators for IoT edge analytics

Throughput Optimization for Storm-based Processing of Stream Data on Clouds.