Abstract:With the deepening of the digital age of information, people's daily data is getting larger and larger, and it is more and more difficult to quantify and process. At this time, the data processing means becomes particularly important. This paper compares and analyzes some methods from data visualization to data dimensionality reduction to outlier detection. In this paper, two different types of datasets, ModelNet40, and red wine quality, are used to introduce the visualization method of the Farthest Point Sampling (FPS). This method can have a clear visual effect on the data dimension and scale, and allow users to observe the structure, type, and scale of the data. In data dimensionality reduction, the study uses Principal Component Analysis (PCA), T-Distributed Stochastic Neighbor Embedding (t-SNE), Triplets Manifold Approximation and Projection (TriMAP), Uniform Manifold Approximation and Projection (UMAP), Pairwise Controlled Manifold Approximation Projection (PaCMAP), and Autoencoder to compare their dimensionality reduction effects. Through these methods, this paper finds that different methods have different effects on different datasets. Therefore, in data dimensionality reduction, it can get twice the result with half the effort by choosing the appropriate method. Finally, this paper also detects outliers. Outliers in datasets will make it difficult for people to process data and make subsequent results inaccurate, so it is necessary to identify outliers. This paper involves methods such as isolation forest and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). Through this paper, the methods of different datasets are analyzed and summarized.

High dimensionality: The latest challenge to data analysis

Tensor Methods in High Dimensional Data Analysis: Opportunities and Challenges

Big data dimensional analysis

Big Data Scaling through Metric Mapping: Exploiting the Remarkable Simplicity of Very High Dimensional Spaces using Correspondence Analysis

Challenges of Big Data Analysis

Statistical Inference from High Dimensional Data

Understanding High Dimensional Spaces through Visual Means Employing Multidimensional Projections

High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

The Data Big Bang and the Expanding Digital Universe: High-Dimensional, Complex and Massive Data Sets in an Inflationary Epoch

High dimensional statistical inference: theoretical development to data analytics

Big Data, Big Challenges

Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery

Big Data: the End of the Scientific Method?

Exploring Big Data Analysis: Fundamental Scientific Problems

Principal component analysis and clustering on manifolds

Digital medicine and the curse of dimensionality

Recent Developments in Complex and Spatially Correlated Functional Data

Various dimension reduction techniques for high dimensional data analysis: a review

Comparison of Data Visualization, Outlier Detection and Data Dimensionality Reduction Methods

Ten quick tips for effective dimensionality reduction

Statistical Methods in Topological Data Analysis for Complex, High-Dimensional Data