Abstract:Due to the rapid development of information technology and network technology, there is a lot of data, but the phenomenon of lack of knowledge is becoming more and more serious. Data mining technology has developed vigorously in this environment, and it has shown more and more vitality. Based on Spark programming model, this paper designs the parallel extension of fuzzy c-means. In order to enhance the performance of fuzzy c-means parallel expansion, the improvement strategy of k-means during the initialization phase is borrowed, and k-means// is extended to fuzzy c-means to obtain better clustering performance. Combined with Spark's programming model, this paper can obtain extended parallel fuzzy c-means algorithm. Several experiments on the data set of the algorithm proposed in this paper have shown good scalability and parallelism, effectively expanding fuzzy c-means clustering to distributed applications, greatly increasing the scale of the data processed by the algorithm. This improves the robustness of the algorithm and the adaptability of the algorithm to the shape and structure of the data, so that the parallel and scalable clustering algorithm can more effectively perform cluster analysis on big data. Three algorithms were simulated on MATLAB platform. We use simple data sets and complex two-dimensional data sets, and compare with the traditional fuzzy c-means algorithm and fuzzy c-means algorithm based on fuzzy entropy. Experiments show that the scalable parallel fuzzy c-means algorithm not only greatly improves the anti-noise performance, but also improves the convergence speed, and it can automatically determine the optimal number of clusters.

A novel distributed machine learning method for classification: parallel covering algorithm

A Parallel SVM Training Algorithm on Large-Scale Classification Problems

An Improved Parallel SVM Algorithm on Distributed System

Parallel Approaches to Neighborhood Rough Sets: Classification and Feature Selection

A Distributed and Scalable Machine Learning Approach for Big Data

A Parallel Varied Density-Based Clustering Algorithm with Optimized Data Partition

Parallel Implementation Of Classification Algorithms Based On Mapreduce

The Scalability for Parallel Machine Learning Training Algorithm: Dataset Matters

Kernel Covering Algorithm for Machine Learning

Covering Algorithm Based on Neighborhood Search and Its Applications

A Novel Parallel Community Detection Scheme Based on Label Propagation

Distributed machine learning: Foundations, trends, and practices

Data Mining Algorithm for Cloud Network Information Based on Artificial Intelligence Decision Mechanism

Cross Covering Algorithm Based on Simulated Annealing

A Novel Online Learning Algorithm Of Support Vector Machines

Parallel Massive Clustering of Discrete Distributions

A Novel Projection Nonparallel Support Vector Machine for Pattern Classification

A PARALLEL AND MODULAR PATTERN CLASSIFICATION FRAMEWORK FOR LARGE-SCALE PROBLEMS

Distributed Classification for Imbalanced Big Data in Distributed Environments

A Parallel Decision Tree Based Algorithm on MPI for Multi-label Classification Learning

Patent Classification Using Parallel Min-Max Modular Support Vector Machine