Abstract:This dissertation deals with developing parallel processing algorithms for Graphic Processing Unit (GPU) in order to solve machine learning problems for large datasets. In particular, it contributes to the development of fast GPU based algorithms for calculating distance (i.e. similarity, affinity, closeness) matrix. It also presents the algorithm and implementation of a fast parallel Support Vector Machine (SVM) using GPU. These application tools are developed using Computing Unified Device Architecture (CUDA), which is a popular software framework for General Purpose Computing using GPU (GPGPU). Distance calculation is the core part of all machine learning algorithms because the closer the query is to some data samples (i.e. observations, records, entries), the more likely the query belongs to the class of those samples. K-Nearest Neighbors (k-NNs) search is a popular and powerful distance based tool for solving classification problem. It is the prerequisite for training local model based classifiers. Fast distance calculation can significantly improve the speed performance of these classifiers and GPUs can be very handy for their accelerations. Meanwhile, several GPU based sorting algorithms are also included to sort the distance matrix and seek for the k-nearest neighbors. The speed performances of the sorting algorithms vary depending upon the input sequences. The GPUKNN proposed in this dissertation utilizes the GPU based distance computation algorithm and automatically picks up the most suitable sorting algorithm according to the characteristics of the input datasets.Every machine learning tool has its own pros and cons. The advantage of SVM is the high classification accuracy. This makes SVM possibly one of the best classifiers. However, as in many other machine learning algorithms, SVM’s training phase slows down when the size of the input dataset increases. The GPU version of parallel SVM based on parallel Sequential Minimal Optimization (SMO) implemented in this dissertation is proposed to reduce the time cost in both training and predicting phases. This implementation of GPUSVM is original. It utilizes many parallel processing techniques to accelerate and minimize the computations of kernel evaluation, which are considered as the most time consuming operations in SVM. Although the manycore architecture of GPU performs the best in data level parallelism, multi-task (aka. task level parallelism) processing is also integrated into the application to improve the speed performance of tasks such as multiclass classification and cross-validation. Furthermore, the procedure of finding worst violators are distributed to multiple blocks on the CUDA model. This reduces the time cost for each iteration of SMO during the training phase. All of these violators are shared among different tasks in multiclass classification and cross-validation to reduce the duplicate kernel computations. The speed performance results have shown that the achieved speedup of both the training phase and predicting phase are ranging from one order of magnitude to three orders of magnitude times faster compared to the state of the art LIBSVM software on some well known benchmarking datasets.

Parallel VINS-Mono Algorithm Based on GPUs in Embedded Devices

High-Performance Visual Odometry with Two-Stage Local Binocular Ba and Gpu

High-Speed Scene Flow on Embedded Commercial Off-the-Shelf Systems.

Realization of CUDA-based Real-Time Multi-Camera Visual SLAM in Embedded Systems

Parallel Pipeline Implementation for Moving Objects Detection in Traffic Video Analysis on a Heterogeneous Platform

GPU based parallel optimization for real time panoramic video stitching

Acceleration of Object Tracking in High‐speed Videogrammetry Using a Parallel OpenMP and SIMD Strategy

Research and Implementation of Parallel Lane Detection Algorithm Based on GPU

GPU-based Parallel Optimization for Real-Time Scale-Invariant Feature Transform in Binocular Visual Registration.

Implementation of Scale and Rotation Invariant On-Line Object Tracking Based on CUDA.

Scalable Parallel Motion Estimation on Muti-GPU System

Parallel Programming Design and Storage Optimization of Remote Sensing Image Registration Based on GPU

Visual Inertial SLAM System Based on Embedded Parallel Processing

Design and Application of Parallel Stereo Matching Algorithm Based on CUDA.

A New Parallel Scheduling Algorithm Based on MPI

Parallel singular value decomposition on heterogeneous multi-core and multi-GPU platforms

An Embedded-GPU-Based Scheme for Real-Time Imaging Processing of Unmanned Aerial Vehicle Borne Video Synthetic Aperture Radar

Design and Storage Optimization of GPU-based Parallel Program of Image Registration for Remote Sensing

Fast parallel machine learning algorithms for large datasets using graphic processing unit

Linear Bregman Algorithm Implemented in Parallel Gpu

GPU based real-time SLAM of six-legged robot