Research on Methods of Dimensionality Reduction in High-dimensional Data

YU Xiao-sheng,ZHOU Ning

DOI: https://doi.org/10.3969/j.issn.1007-7634.2007.08.029

2007-01-01

Abstract:In the paper the authors introduce three key methods of dimensionality reduction in high-dimensional dataset,such as MDS,Isomap.At the same time the authors discuss applications of those methods.

What problem does this paper attempt to address?

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

赵武锋,沈海斌,严晓浪

DOI: https://doi.org/10.3785/j.issn.1008-9497.2009.06.015

2009-01-01

Abstract:In the case of small sample size,classical linear discriminant analysis fails due to the singularity of scatter matrices for supervised dimensionality reduction.Many extensions were proposed in the past to overcome this problem,which in general can be classified into three categories: methods based on the null space of the within-class scatter matrix,ones based on the range space of the total scatter matrix and ones based on other subspace.In order to better understand the characteristics of the algorithms of the former two classes,a computational and theoretical analysis was carried out,and concluded that: under a mild condition which holds in many applications involving high-dimensional data,methods based on the null space of the within-class scatter matrix are equivalent to those based on the range space of the total scatter matrix except two differences of constraints for discriminant vectors and implementation procedure.The comparative results on the face database,ORL and YALE,also confirmed the aforementioned conclusion.
Research on Methods of High - Dimensional Data Visualization

YU Xiao-sheng,ZHOU Ning,ZHANG Fang-fang

DOI: https://doi.org/10.3969/j.issn.1007-7634.2007.01.024

2007-01-01

Abstract:In this paper the authors discuss three basic elements of successfully visualizing high-dimensional data.At the same time the authors introduce several typical high-dimensional data visualization methods.
Survey of Dimension Reduction Methods in High-dimensional Indexing Schemes

WU Ling-da,HE Ling,CAI Yi-chao

DOI: https://doi.org/10.3969/j.issn.1001-3695.2006.12.002

2006-01-01

Abstract:Gives a comprehensive summarization of existing dimension reduction approaches,analyzes their merits,compares their shortcomings with each other and points out the future direction of them from the application angle,which is the syncretizing of various techniques,especially the combination of linear dimension reduction with non-linear dimension reduction techniques.
Research and Comparison of Data Dimensionality Reduction Algorithms

Qin Liu,Ran Chen,Hongming Zhu,Hongfei Fan

DOI: https://doi.org/10.1145/3135954.3135965

2017-01-01

Abstract:With the explosive growth of information data, there are more and more situations in the academic research and industrial fields where a large amount of high-dimension data to be dealt with. Consequently, this has induced tremendous difficulties for noise processing and information mining. The dimensionality reduction algorithms are playing an increasingly significant role in the process of dealing with high dimensional data. A large number of data dimensionality reduction methods have been proposed in recent years. These data reduction algorithms try to find the internal connection within data, reduce the scale of data and retain the original data information. These algorithms conduct data dimensionality reduction from varied perspectives including mapping, and data similarity, etc. As a result, there are discrepancies among the said algorithms in terms of applicable targets, computational complexity and retention of the original information. In this paper, the principle of the existing typical dimensionality reduction algorithms is expounded, and the time complexity, data information retention and the object of application are compared via experiments. The remaining problems of data reduction are summarized in the end.
Research on Dimensionality Reduction Technology of Hyperspectral Data

WANG Xu-hong,XIAO-ping,GUO Jian-ming

DOI: https://doi.org/10.3969/j.issn.1000-288x.2006.06.022

2006-01-01

Abstract:Hyperspectral data have a high spectral resolution for the objects of the earth. However,many analysis approaches of hyperspectral data do not provide a promising result because of its great data volume and strong correlation between its neighboring bands.Consequently,it restricts the efficiency and broad application of high resolution data.The research indicates that feature extraction is the highly effective theory and method to optimize hyperspectral data and information.The result of experiment shows that with a given precision of classification,the reduction in dimensionality without loss of information improves the classifier performance,and helps to achieve the aims of optimal process and effective utilization of hyperspectral remote sensing data.
The Dimension Reduction Method Based on Data Transformation

WU Xinling,WU Guoqing

2006-01-01

Abstract:The data dimension reduction is the main method that can enhance the data mining efficiency based on higher-dimension data set.In this paper,the method and the application using data transformation to reduce data dimension are studied.A general data transformation model is proposed.How to compute the data transformation matrix of the model with the principal component analysis method is introduced explicitly.And a application example about this method is given too.The example indicates that (it's) possible to capture the maximum variations of the original variables with fewer variables.
Feature dimensionality reduction: a review

Weikuan Jia,Meili Sun,Jian Lian,Sujuan Hou

DOI: https://doi.org/10.1007/s40747-021-00637-x

2022-01-21

Abstract:Abstract As basic research, it has also received increasing attention from people that the “curse of dimensionality” will lead to increase the cost of data storage and computing; it also influences the efficiency and accuracy of dealing with problems. Feature dimensionality reduction as a key link in the process of pattern recognition has become one hot and difficulty spot in the field of pattern recognition, machine learning and data mining. It is one of the most challenging research fields, which has been favored by most of the scholars’ attention. How to implement “low loss” in the process of feature dimension reduction, keep the nature of the original data, find out the best mapping and get the optimal low dimensional data are the keys aims of the research. In this paper, two-dimensionality reduction methods, feature selection and feature extraction, are introduced; the current mainstream dimensionality reduction algorithms are analyzed, including the method for small sample and method based on deep learning. For each algorithm, examples of their application are given and the advantages and disadvantages of these methods are evaluated.

computer science, artificial intelligence
Dimensionality reduction based on spectral graph and its application

WAN Hai-ping,HE Hua-can

2006-01-01

Abstract:Most machine learning tasks confront the problem of dimensionality reduction for extracting meaningful features and processing convenience.In these topological spaces,it usually adopts Euclidean distance to measure similarity between objects.It is argued that in many learning tasks the path from one object to another will also be a proper alternative.Also the relationship between local and global information is discussed when selecting features.A dimensionality reduction method incorporating both path and distance feature is proposed based on spectral graph theory which aims at preserving local meaningful neighborhood structures in the original data.In the experiments of both face recognition and information retrieval,it achieves positive results.
Dimensionally Reduction: An Experimental Study

Xin Song,Zhihua Cai,Wei Xu

DOI: https://doi.org/10.1007/978-3-319-14717-8_30

2014-01-01

Abstract:Dimensionality reduction as the available method to overcome the “curses of dimensionality” has attracted wide attention. However, the pervious studies treat the visualization and the subsequent classification performance separately. In this case, we do not know whether there is a underlying relationship (i.e., direct proportion) between visualization and the followed classification performance.
Overview and comparative study of dimensionality reduction techniques for high dimensional data

Shaeela Ayesha,Muhammad Kashif Hanif,Ramzan Talib

DOI: https://doi.org/10.1016/j.inffus.2020.01.005

IF: 18.6

2020-07-01

Information Fusion

Abstract:<p>The recent developments in the modern data collection tools, techniques, and storage capabilities are leading towards huge volume of data. The dimensions of data indicate the number of features that have been measured for each observation. It has become a challenging task to analyze high dimensional data. Different dimensionality reduction techniques are available in literature to eliminate irrelevant and redundant features. Selection of an appropriate dimension reduction technique can help to enhance the processing speed and reduce the time and effort required to extract valuable information. This paper presents the state-of-the art dimensionality reduction techniques and their suitability for different types of data and application areas. Furthermore, the issues of dimensionality reduction techniques have been highlighted that can affect the accuracy and relevance of results.</p>

computer science, artificial intelligence, theory & methods
Supervised nonlinear dimensionality reduction for visualization and classification

GengXin,ZhanDe-Chuan,ZhouZhi-Hua

2005-01-01

Abstract:When performing visualization and classification, people often confront the problem of dimensionality reduction. Isomap is one of the most promising nonlinear dimensionality reduction techniques. H...
A Brief Survey of Dimension Reduction

Li Song,Hongbin Ma,Mei Wu,Zilong Zhou,Mengyin Fu

DOI: https://doi.org/10.1007/978-3-030-02698-1_17

2018-01-01

Abstract:Dimension reduction problem is a big concern which can reduce the scale of a database and keep the main features of these data simultaneously. This paper aims at reviewing and comparing different dimension reduction algorithms. Mainly, the performances of four basic algorithms (PCA, LDA, LLE and LE), their improved methods and deep learning methods are compared by reviewing the previous work. Their recognition accuracy and running time are carefully analyzed. We conclude that PCA and LDA are used more frequently in related fields. Combined methods usually perform better than original methods. Besides, deep learning method is also an approach developed in recent years, which outperforms existing traditional algorithms, though there are many barriers at present, such as obtaining huge labeled database, the computing and power limitation of different systems etc. Future research should focus on the processing of larger database. Finally, some new applications of dimension reduction are reviewed.
Real-valued Multivariate Dimension Reduction: A Survey

Hong-Ming SHAN,Jun-Ping ZHANG

DOI: https://doi.org/10.16383/j.aas.2018.c160812

2018-01-01

Abstract:As one of the classical problems in machine learning,dimension reduction is used for dealing with the curse of dimensionality,speeding up computational efficiency of the algorithm,and improving interpretability as well as visualizing high-dimensional data.Traditional dimension reduction algorithms such as principal component analysis (PCA) and linear discriminant analysis are mainly suitable for unlabeled data or classification data.When the response variables are univariate or multivariate continuous real-valued ones,however,such dimension reduction methods cannot guarantee the effective predictive performance of the reduced subspace.In the recent two decades,researchers have been devoted to studying this issue with different viewpoints,attaining many promising and systemic achievements.Under this background,we will survey the developments of real-valued multivariate dimension reduction in detail.We will also introduce its basic concepts,algorithms and theories,and discuss some potential research directions deserving investigating.
Linear Low-Rank Approximation and Nonlinear Dimensionality Reduction

Zhenyue Zhang,Hongyuan Zha

DOI: https://doi.org/10.1360/04ys0016

2004-01-01

Science in China Series A Mathematics

Abstract:We present our recent work on both linear and nonlinear data reduction methods and algorithms: for the linear case we discuss results on structure analysis of SVD of columnpartitioned matrices and sparse low-rank approximation; for the nonlinear case we investigate methods for nonlinear dimensionality reduction and manifold learning. The problems we address have attracted great deal of interest in data mining and machine learning.
Data Dimension Reduction Based on Category Theory

ZHOU Li-li,LI Fan-zhang

DOI: https://doi.org/10.3969/j.issn.1002-137x.2011.09.057

2011-01-01

Computer Science

Abstract:The research of category theory is mainly about summary and abstraction of some specific mathematical objects and mapping.The dimensionality reduction problem based on category theory was discussed,and it could solve the problems of image analysis and image recognition.Besides,the process of dimensionality reduction based on category theory was listed.There are two examples,Principal Component Analysis Category and Isomap Category,which verified the correctness of the category theory applied to the dimensionality reduction problem.
Dimensionality Reduction and Data Visualisation

Hujun Yin

2008-01-01

Abstract:Dimension reduction has long been associated with retinotopic mapping for understanding cortical maps and neural information processing. Multisensory information is perceived, propagated and mapped onto the 2-D cortex in a near-optimal information preserving manner. Data visualization, inspired by this mechanism, is playing an increasingly important role in many applications involving feature and data reduction, from biology, neuroscience, decision support, social science, to management science. The topic has also attracted a great deal of attention in computer vision and pattern recognition. Classic linear methods include principal component analysis (PCA), factor analysis, projection pursuit and independent component analysis. Recently there have been considerable efforts and advances in developing methodologies and techniques for nonlinear dimensionality reduction. A number of novel projection methods have been proposed from statistics, geometry theory and neural networks. Two fundamental approaches are multidimensional scaling and nonlinear PCA. This tutorial will provide an introduction to this challenging and demanding topic. Various methods along these lines such as, self-organising maps, kernel PCA, principal manifold, metric and non-metric scaling, isomap, local linear embedding, Laplacian eigenmap, as well as spectral clustering will be explained and discussed. It will also attempt to unify these methods under a constrained self-organising framework. Examples and applications will be shown to illustrate the usefulness and strengthen of various methods, as well their weakness and limitation.
Comparison of Data Visualization, Outlier Detection and Data Dimensionality Reduction Methods

Xingyu Zhao

DOI: https://doi.org/10.54097/wgchmc87

2024-03-13

Abstract:With the deepening of the digital age of information, people's daily data is getting larger and larger, and it is more and more difficult to quantify and process. At this time, the data processing means becomes particularly important. This paper compares and analyzes some methods from data visualization to data dimensionality reduction to outlier detection. In this paper, two different types of datasets, ModelNet40, and red wine quality, are used to introduce the visualization method of the Farthest Point Sampling (FPS). This method can have a clear visual effect on the data dimension and scale, and allow users to observe the structure, type, and scale of the data. In data dimensionality reduction, the study uses Principal Component Analysis (PCA), T-Distributed Stochastic Neighbor Embedding (t-SNE), Triplets Manifold Approximation and Projection (TriMAP), Uniform Manifold Approximation and Projection (UMAP), Pairwise Controlled Manifold Approximation Projection (PaCMAP), and Autoencoder to compare their dimensionality reduction effects. Through these methods, this paper finds that different methods have different effects on different datasets. Therefore, in data dimensionality reduction, it can get twice the result with half the effort by choosing the appropriate method. Finally, this paper also detects outliers. Outliers in datasets will make it difficult for people to process data and make subsequent results inaccurate, so it is necessary to identify outliers. This paper involves methods such as isolation forest and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). Through this paper, the methods of different datasets are analyzed and summarized.
An Adaptive and Dynamic Dimensionality Reduction Method for High-Dimensional Indexing.

Heng Tao Shen,Xiaofang Zhou,Aoying Zhou

DOI: https://doi.org/10.1007/s00778-005-0167-3

2005-01-01

The VLDB Journal

Abstract:The notorious "dimensionality curse" is a well-known phenomenon for any multi-dimensional indexes attempting to scale up to high dimensions. One well-known approach to overcome degradation in performance with respect to increasing dimensions is to reduce the dimensionality of the original dataset before constructing the index. However, identifying the correlation among the dimensions and effectively reducing them are challenging tasks. In this paper, we present an adaptive Multi-level Mahalanobis-based Dimensionality Reduction (MMDR) technique for high-dimensional indexing. Our MMDR technique has four notable features compared to existing methods. First, it discovers elliptical clusters for more effective dimensionality reduction by using only the low-dimensional subspaces. Second, data points in the different axis systems are indexed using a single B+-tree. Third, our technique is highly scalable in terms of data size and dimension. Finally, it is also dynamic and adaptive to insertions. An extensive performance study was conducted using both real and synthetic datasets, and the results show that our technique not only achieves higher precision, but also enables queries to be processed efficiently.
Recent Advances In Supervised Dimension Reduction: A Survey

Guoqing Chao,Yuan Luo,Weiping Ding

DOI: https://doi.org/10.3390/make1010020

2019-01-01

Machine Learning and Knowledge Extraction

Abstract:Recently, we have witnessed an explosive growth in both the quantity and dimension of data generated, which aggravates the high dimensionality challenge in tasks such as predictive modeling and decision support. Up to now, a large amount of unsupervised dimension reduction methods have been proposed and studied. However, there is no specific review focusing on the supervised dimension reduction problem. Most studies performed classification or regression after unsupervised dimension reduction methods. However, we recognize the following advantages if learning the low-dimensional representation and the classification/regression model simultaneously: high accuracy and effective representation. Considering classification or regression as being the main goal of dimension reduction, the purpose of this paper is to summarize and organize the current developments in the field into three main classes: PCA-based, Non-negative Matrix Factorization (NMF)-based, and manifold-based supervised dimension reduction methods, as well as provide elaborated discussions on their advantages and disadvantages. Moreover, we outline a dozen open problems that can be further explored to advance the development of this topic.
Study on the Low-Dimensional Embedding and the Embedding Dimensionality of Manifold of High-Dimensional Data

ZHAO Lian-wei,LUO Si-wei,ZHAO Yan-Chang,LIU Yun-hui

DOI: https://doi.org/10.1360/jos161423

2005-01-01

Journal of Software

Abstract:Finding meaningful low-dimensional embedded in a high-dimensional space is a classical problem. Isomap is a nonlinear dimensionality reduction method proposed and based on the theory of manifold. It not only can reveal the meaningful low-dimensional structure hidden in the high-dimensional observation data, but can recover the underlying parameter of data lying on a low-dimensional submanifold. Based on the hypothesis that there is an isometric mapping between the data space and the parameter space, Isomap works, but this hypothesis has not been proved. In this paper, the existence of isometric mapping between the manifold in the high-dimensional data space and the parameter space is proved. By distinguishing the intrinsic dimensionality of high-dimensional data space from the manifold dimensionality, and it is proved that the intrinsic dimensionality is the upper bound of the manifold dimensionality in the high-dimensional space in which there is a toroidal manifold. Finally an algorithm is proposed to find the underlying toroidal manifold and judge whether there exists one. The results of experiments on the multi-pose three-dimensional object show that the method is effective.

Research on Methods of Dimensionality Reduction in High-dimensional Data

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Research on Methods of High - Dimensional Data Visualization

Survey of Dimension Reduction Methods in High-dimensional Indexing Schemes

Research and Comparison of Data Dimensionality Reduction Algorithms

Research on Dimensionality Reduction Technology of Hyperspectral Data

The Dimension Reduction Method Based on Data Transformation

Feature dimensionality reduction: a review

Dimensionality reduction based on spectral graph and its application

Dimensionally Reduction: An Experimental Study

Overview and comparative study of dimensionality reduction techniques for high dimensional data

Supervised nonlinear dimensionality reduction for visualization and classification

A Brief Survey of Dimension Reduction

Real-valued Multivariate Dimension Reduction: A Survey

Linear Low-Rank Approximation and Nonlinear Dimensionality Reduction

Data Dimension Reduction Based on Category Theory

Dimensionality Reduction and Data Visualisation

Comparison of Data Visualization, Outlier Detection and Data Dimensionality Reduction Methods

An Adaptive and Dynamic Dimensionality Reduction Method for High-Dimensional Indexing.

Recent Advances In Supervised Dimension Reduction: A Survey

Study on the Low-Dimensional Embedding and the Embedding Dimensionality of Manifold of High-Dimensional Data