Abstract:Support Vector Machine (SVM) regression is an important technique in data mining. The SVM training is expensive and its cost is dominated by: (i) the kernel value computation, and (ii) a search operation which finds extreme training data points for adjusting the regression function in every training iteration. Existing training algorithms for SVM regression are not scalable to large datasets because: (i) each training iteration repeatedly performs expensive kernel value computations, which is inefficient and requires holding the whole training dataset in memory; (ii) the search operation used in each training iteration considers the whole search space which is very expensive. In this article, we significantly improve the scalability and efficiency of SVM regression by exploiting the high performance of Graphics Processing Units (GPUs) and solid state drives (SSDs). Our key ideas are as follows. (i) To reduce the cost of repeated kernel value computations and avoid holding the whole training dataset in the GPU memory, we precompute all the kernel values and store them in the CPU memory extended by the SSD; together with an efficient strategy to read the precomputed kernel values, reusing precomputed kernel values with an efficient retrieval is much faster than computing them on-the-fly. This also alleviates the restriction that the training dataset has to fit into the GPU memory, and hence makes our algorithm scalable to large datasets, especially for large datasets with very high dimensionality. (ii) To enhance the performance of the frequently used search operation, we design an algorithm that minimizes the search space and the number of accesses to the GPU global memory; this optimized search algorithm also avoids branch divergence (one of the causes for poor performance) among GPU threads to achieve high utilization of the GPU resources. Our proposed techniques together form a scalable solution to the SVM regression which we call SIGMA. Our extensive experimental results show that SIGMA is highly efficient and can handle very large datasets which the state-of-the-art GPU-based algorithm cannot handle. On the datasets of size that the state-of-the-art GPU-based algorithm can handle, SIGMA consistently outperforms the state-of-the-art GPU-based algorithm by an order of magnitude and achieves up to 86 times speedup.

Parallel Algorithm of Improved FunkSVD Based on GPU

A social network-aware top-N recommender system using GPU.

Improved Algorithm of Incremental Singular Value Decomposition Collaborative Filtering

A Novel Fully Hardware-Implemented SVD Solver Based on Ultra-Parallel BCV Jacobi Algorithm

Incorporating Community Detection and Clustering Techniques into Collaborative Filtering Model

Parallel Inference for Latent Dirichlet Allocation on Graphics Processing Units.

SVD++ Recommendation Algorithm Based on Backtracking

Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark

Scalable and Fast SVM Regression Using Modern Hardware.

Accelerating large partial EVD/SVD calculations by filtered block Davidson methods

Practical Lossless Federated Singular Vector Decomposition over Billion-Scale Data

MASCOT: Fast and Highly Scalable SVM Cross-Validation Using GPUs and SSDs

Research on big data mining based on improved parallel collaborative filtering algorithm

An FPGA-Based Accelerator for Neighborhood-Based Collaborative Filtering Recommendation Algorithms

Implementation of Video Abstract Algorithm Based on CUDA

cuFSDAF: An Enhanced Flexible Spatiotemporal Data Fusion Algorithm Parallelized Using Graphics Processing Units

A GPU-accelerated Non-Negative Sparse Latent Semantic Analysis Algorithm for Social Tagging Data

A Novel Learning Rate Function and Its Application on the SVD++ Recommendation Algorithm

CuMF_SGD: Parallelized Stochastic Gradient Descent for Matrix Factorization on GPUs.

A GPU-based high-throughput image retrieval algorithm

GPU-accelerated Faster Mean Shift with euclidean distance metrics