Abstract:The maximal frequent itemsets issue in big data processing has become a hot research topic. Most of the previous work on big data processing directly analyzes the data through the existing approaches, which would cause problems of redundant computation, high time complexity, and large storage space. To solve the problems, this paper proposes a Heuristic MapReduce-based Association rule approach through Maximal frequent itemsets mining, HMAM. The main idea is: At first, by directly operating on the transaction database, we allocate transactions to different processing nodes and group all transactions according to dimension. Then, we screen the most frequent transactions from each transaction set using the Bitmap-Sort and obtain best-transaction-set through aggregating all the transaction-elects of each transaction set. The current candidate maximal frequent itemsets can be acquired by removing sub-transactions in terms of the inclusion relations of the items in best-transaction-set. At the same time, each subset of sub-transactions in the candidate maximal frequent itemsets is discarded from all transaction sets. Then the final candidate maximal frequent itemsets can be obtained by iteration until each transaction set is empty. Finally, we achieve the acquisition of maximal frequent itemsets by employing the minimum support threshold. The experimental results demonstrate that compared with the existing approaches, HMAM significantly avoids producing a large number of candidate itmesets resulting from join operation, accelerates the speed of mining the maximal frequent itemsets, and improves the utilization rate of resources simultaneously.

Detecting Associations in Large Dataset on MapReduce

Distributed Affinity Propagation Clustering Based on MapReduce

SuperMIC: Analyzing Large Biological Datasets in Bioinformatics with Maximal Information Coefficient

Detecting novel multi-variable associations in big data based on MIC

Parallel Graph Pattern Matching in Massive Networks Based on MapReduce

Detecting Three‐Dimensional Associations in Large Data Set

Using Traditional Data Analysis Algorithms To Detect Access Patterns For Big Data Processing

Complex statistical analysis of big data: Implementation and application of Apriori and FP-Growth algorithm based on MapReduce

Large-scale Data Mining Method based on Clustering Algorithm Combined with MAPREDUCE

A Parallel Implementation of the K-Means Algorithm Based on MapReduce

An improved parallel association rules algorithm based on MapReduce framework for big data

MR-Mafia: Parallel Subspace Clustering Algorithm Based on MapReduce for Large Multi-dimensional Datasets

A Novel Parallel Algorithm for Frequent Itemsets Mining in Massive Small Files Datasets

Parallel Simrank Computing on Large Scale Dataset on Mapreduce

LI-MR: A Local Iteration Map/Reduce Model and Its Application to Mine Community Structure in Large-Scale Networks

MapReduce As a Programming Model for Association Rules Algorithm on Hadoop

A Parallel Computing Model for Large-Graph Mining with MapReduce.

A Novel Process-Based Association Rule Approach Through Maximal Frequent Itemsets for Big Data Processing.

Parallel Implementation for Co-occurrence Statistics with MapReduce Model

Efficient Large-Scale Multi-graph Similarity Search Using MapReduce.

Parallel Mining Algorithm of Association Rules on Massive Data