Abstract:SpeedStream is a universal distributed platform that can handle with massive data flows with the features of low coupling, high availability, low latency and high scalability. Focusing on the core technologies of real-time stream computing platform in cloud environment, this paper conducts a series of researches and implementation of the system. First of all, aiming at the availability of real-time streaming computing platform, we design a high availability framework based on Zookeeper. It ensures fault detection and recovery of process level and node level timely by monitoring heartbreak of each modules and strategy of fault migration. Secondly, in order to increase the application types of the platform, by means of directed cycle detection and iteration protection, we design a real-time streaming computing model that based on directed graph with sources and sinks, which can not only satisfy the needs of common DAG computing services, but also support iteration computing services including directed cycle, bidirectional arcs and annular arcs. In addition, the platform can realize personalized task scheduling strategy for users by establishing task allocation matrix and optimize task allocation model. Finally, in order to solve the many-to-many dynamic load-balancing between tasks, we apply scheduler with status and distributed session table. It overcomes the difficulty of maintaining consistency of session without global session table. We also testified the convergence of this method. The experiment indicates that the throughput and data processing delay of SpeedStream are superior to other alternatives in dealing with the businesses of iteration applications, high traffic fluctuation applications, and high demand of load-balancing applications. This platform provides reliable, universal, and real-time solutions to process massive data flows, such as to process the real-time trading data in e-commerce, to analyze sensing flow in internet of things, and monitor traffics of the Internet.

Elastic Stream Cloud (ESC): A Stream-Oriented Cloud Computing Platform for Rich Internet Application

Esc: Towards an Elastic Stream Computing Platform for the Cloud

SSC: Cloud-Based Data Stream Management in Distributed Environments

Elastic stream processing in the Cloud

Elastic Stream Processing for Distributed Environments.

Study and Implementation of Elastic Stream Computing in the Cloud

Emotion-Based Social Computing Platform for Streaming Big-Data: Architecture and Application

SpeedStream: A real-time stream data processing platform in the cloud.

An Elastic Scalable Stream Processing Framework Based on Actor Model

Cloudets: Cloud-based Cognition for Large Streaming Data

Coping With Heterogeneous Video Contributors and Viewers in Crowdsourced Live Streaming: A Cloud-Based Approach

A Cloud Computing Application in Land Resources Information Management

Resource-Aware Stream Processing in High Performance Cloud Environment

Block-Stream As a Service: A More Secure, Nimble, and Dynamically Balanced Cloud Service Model for Ambient Computing

Cloudet: A Cloud-Driven Visual Cognition of Large Streaming Data.

Elastic Resource Provisioning For Batched Stream Processing System In Container Cloud

Streamforce: outsourcing access control enforcement for stream data to the clouds

Edge Cloud Collaborative Stream Computing for Real-Time Structural Health Monitoring

Migrating Live Streaming Applications Onto Clouds: Challenges and a CloudStorm Solution

A Framework for Partitioning and Execution of Data Stream Applications in Mobile Cloud Computing

Master: Multi-Platform Application Streaming Toolkits For Elastic Resources