Abstract:Data mining is the process used for extracting hidden patterns from large databases using a variety of techniques. For example, in supermarkets, we can discover the items that are often purchased together and that are hidden within the data. This helps make better decisions which improve the business outcomes. One of the techniques that are used to discover frequent patterns in large databases is frequent itemset mining (FIM) that is a part of association rule mining (ARM). There are different algorithms for mining frequent itemsets. One of the most common algorithms for this purpose is the Apriori algorithm that deduces association rules between different objects which describe how these objects are related together. It can be used in different application areas like market basket analysis, student’s courses selection process in the E-learning platforms, stock management, and medical applications. Nowadays, there is a great explosion of data that will increase the computational time in the Apriori algorithm. Therefore, there is a necessity to run the data-intensive algorithms in a parallel-distributed environment to achieve a convenient performance. In this paper, optimization of the Apriori algorithm using the Spark-based cuckoo filter structure (ASCF) is introduced. ASCF succeeds in removing the candidate generation step from the Apriori algorithm to reduce computational complexity and avoid costly comparisons. It uses the cuckoo filter structure to prune the transactions by reducing the number of items in each transaction. The proposed algorithm is implemented on the Spark in-memory processing distributed environment to reduce processing time. ASCF offers a great improvement in performance over the other candidate algorithms based on Apriori, where it achieves a time of only 5.8% of the state-of-the-art approach on the retail dataset with a minimum support of 0.75%.

Research on Logistics Path Frequent Patterns Based on Parallel Apriori

Data analysis in real-time supply chain based on frequent pattern mining

Research on Association Rules Mining Algorithm Based on Hadoop-Taking Apriori as an Example

An Efficient Frequent Patterns Mining Algorithm Based on Apriori Algorithm and the Fp-Tree Structure

A Method to Optimize Apriori Algorithm for Frequent Items Mining

Analysis of Key Commuting Routes Based on Spatiotemporal Trip Chain

Research on Mining Frequent Path and Prediction Algorithms of Object Movement Patterns in Rfid Database

Using Apriori to mine IoT frequent structures on compute cloud

Research and Analysis of Fast Apriori Algorithm in Single Dimensional Association Rules

A MapReduce-Based Parallel Frequent Pattern Growth Algorithm for Spatiotemporal Association Analysis of Mobile Trajectory Big Data

Association Rule Mining Based on Apriori Algorithm in Distribution Center Storage Planning

Complex statistical analysis of big data: Implementation and application of Apriori and FP-Growth algorithm based on MapReduce

Mining Urban Moving Tra jectory Patterns Based on Multi-scale Space Partition and Road Network Modeling

Parallel Graph Pattern Matching in Massive Networks Based on MapReduce

A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data

An improved parallel association rules algorithm based on MapReduce framework for big data

Parallel Apriori Algorithm Based on the Thread Pool

Research on Parallel Association Rules Mining Algorithm Based on Hadoop

Parallel mining and application of fuzzy association rules

ASCF: Optimization of the Apriori Algorithm Using Spark-Based Cuckoo Filter Structure

Improving frequent patterns mining by LFP