Abstract:MapReduce has been widely used for large-scale data analysis in the Cloud. The system is well recognized for its elastic scalability and fine-grained fault tolerance although its performance has been noted to be suboptimal in the database context. According to a recent study [19], Hadoop, an open source implementation of MapReduce, is slower than two state-of-the-art parallel database systems in performing a variety of analytical tasks by a factor of 3.1 to 6.5. MapReduce can achieve better performance with the allocation of more compute nodes from the cloud to speed up computation; however, this approach of "renting more nodes" is not cost effective in a pay-as-you-go environment. Users desire an economical elastically scalable data processing system, and therefore, are interested in whether MapReduce can offer both elastic scalability and efficiency. In this paper, we conduct a performance study of MapReduce (Hadoop) on a 100-node cluster of Amazon EC2 with various levels of parallelism. We identify five design factors that affect the performance of Hadoop, and investigate alternative but known methods for each factor. We show that by carefully tuning these factors, the overall performance of Hadoop can be improved by a factor of 2.5 to 3.5 for the same benchmark used in [19], and is thus more comparable to that of parallel database systems. Our results show that it is therefore possible to build a cloud data processing system that is both elastically scalable and efficient.

A MapReduce-Based Architecture for Rule Matching in Production System

An efficient MapReduce-based rule matching method for production system

A distributed architecture for rule engine to deal with big data

A Novel Approach to Distributed Rule Matching and Multiple Firing Based on MapReduce

MapReduce-based scalable architecture for implementing SVM in hardware

Classified Fuzzy Association Rules Mining Model and Its Application in Aromatic Hydrocarbon Extraction

Performance Evaluation for Distributed Join Based on MapReduce.

Research on Rule Matching Model Based on Spark

Experience Report State-Replication-Based Matching System

A parallel approximate rule extracting algorithm based on the improved discernibility matrix

The Improvement Research on Rule Matching Algorithm Rete in Electronic Commerce Application Systems

Rete Algorithm:Current Issues and Future Challenge

An Improved Rete Algorithm Based on Double Hash Filter and Node Indexing for Distributed Rule Engine

The performance of MapReduce: an in-depth study

The Performance of MapReduce

A parallel fuzzy rule-base based decision tree in the framework of map-reduce

An improved parallel association rules algorithm based on MapReduce framework for big data

A Distributed Rule Engine for Streaming Big Data

On Efficient Matching of Spatiotemporal Rules.