Lecture notes on high-dimensional data

Sven-Ake Wegner

2024-01-28

Abstract:These are lecture notes based on the first part of a course on 'Mathematical Data Science' that I taught to final year BSc students in the UK between 2019 and 2020. Topics include: Concentration of measure in high dimensions; Gaussian random vectors in high dimensions; Random projections; Separation/disentangling of Gaussian data.

Functional Analysis

What problem does this paper attempt to address?

Lecture Notes on "Random Matrices"

Roland Speicher

DOI: https://doi.org/10.48550/arXiv.2009.05157

2020-09-11

Abstract:This in an introduction to random matrix theory, giving an impression of some of the most important aspects of this modern subject. In particular, it covers the basic combinatorial and analytic theory around Wigner's semicircle law, featuring also concentration phenomena, and the Tracy-Widom distribution of the largest eigenvalue. The circular law and a discussion of Voiculescu's multivariate extension of the semicircle law, as an appetizer for free probability theory, also make an appearance. The material here was presented in the winter term 2019/20 at Saarland University in 24 lectures of 90 minutes each. The lectures were recorded and can be found online at <a class="link-external link-https" href="https://www.math.uni-sb.de/ag/speicher/web_video/index.html" rel="external noopener nofollow">this https URL</a>.

Probability,Combinatorics,Operator Algebras
High-dimensional non-convex landscapes and gradient descent dynamics

Tony Bonnaire,Davide Ghio,Kamesh Krishnamurthy,Francesca Mignacco,Atsushi Yamamura,Giulio Biroli

DOI: https://doi.org/10.1088/1742-5468/ad2929

2024-10-31

Journal of Statistical Mechanics Theory and Experiment

Abstract:In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Our aim is to show how approaches developed in physics, mainly statistical physics of disordered systems, can be used to tackle open questions on high-dimensional dynamics in machine learning.

physics, mathematical,mechanics
Mathematics of Neural Networks (Lecture Notes Graduate Course)

Bart M. N. Smets

2024-03-06

Abstract:These are the lecture notes that accompanied the course of the same name that I taught at the Eindhoven University of Technology from 2021 to 2023. The course is intended as an introduction to neural networks for mathematics students at the graduate level and aims to make mathematics students interested in further researching neural networks. It consists of two parts: first a general introduction to deep learning that focuses on introducing the field in a formal mathematical way. The second part provides an introduction to the theory of Lie groups and homogeneous spaces and how it can be applied to design neural networks with desirable geometric equivariances. The lecture notes were made to be as self-contained as possible so as to accessible for any student with a moderate mathematics background. The course also included coding tutorials and assignments in the form of a set of Jupyter notebooks that are publicly available at https://gitlab.com/bsmetsjr/mathematics_of_neural_networks.

Artificial Intelligence,Machine Learning
Lecture Notes on Free Probability

Vladislav Kargin

DOI: https://doi.org/10.48550/arXiv.1305.2611

2013-05-13

Abstract:Lecture notes for a graduate course lectured at Stanford University.

Probability
High-Dimensional Data Analysis with Low-Dimensional Models

John Wright,Yi Ma

DOI: https://doi.org/10.1017/9781108779302

2022-01-01

Abstract:Connecting theory with practice, this systematic and rigorous introduction covers the fundamental principles, algorithms and applications of key mathematical models for high-dimensional data analysis. Comprehensive in its approach, it provides unified coverage of many different low-dimensional models and analytical techniques, including sparse and low-rank models, and both convex and non-convex formulations. Readers will learn how to develop efficient and scalable algorithms for solving real-world problems, supported by numerous examples and exercises throughout, and how to use the computational tools learnt in several application contexts. Applications presented include scientific imaging, communication, face recognition, 3D vision, and deep networks for classification. With code available online, this is an ideal textbook for senior and graduate students in computer science, data science, and electrical engineering, as well as for those taking courses on sparsity, low-dimensional structures, and high-dimensional data. Foreword by Emmanuel Candès.
Lecture notes on the Fundamental Structures of Differential Geometry

Dmitrii Pedchenko

DOI: https://doi.org/10.48550/arXiv.2204.01247

2022-04-04

Abstract:This is the first part of the lecture notes that grew out of the special course given during the 2021-2022 academic year. In these lecture notes we present an approach to the fundamental structures of differential geometry that uses the vernacular of sheaves, differential operators and horizontal subbundles.

Differential Geometry
Algorithms for Massive Data -- Lecture Notes

Nicola Prezza

2024-03-26

Abstract:These are the lecture notes for the course CM0622 - Algorithms for Massive Data, Ca' Foscari University of Venice. The goal of this course is to introduce algorithmic techniques for dealing with massive data: data so large that it does not fit in the computer's memory. There are two main solutions to deal with massive data: (lossless) compressed data structures and (lossy) data sketches. These notes cover both topics: compressed suffix arrays, probabilistic filters, sketching under various metrics, Locality Sensitive Hashing, nearest neighbour search, algorithms on streams (pattern matching, counting).

Data Structures and Algorithms
Lecture notes on signal processing in high dimensional space

qun wan,yang liu,lin zou,j h yin

DOI: https://doi.org/10.1109/TALE.2015.7386029

2015-01-01

Abstract:In the course of signal and information processing, the linear processing method is easier to master than the nonlinear process. However, in practical application, the nonlinear problem is often encountered, which is too complex to be limited by the existing signal and information processing resources. Therefore, it is necessary to transform the complex nonlinear problem into a simple linear problem. According to teaching practice of signal processing course, the linear correlation characteristics of signal in high dimensional space analysis are discussed in this paper. From the perspective of information theory, the advantages of high dimensional spatial analysis and processing of signal and information are also discussed. The results of this paper can provide a new perspective for students to understand why we analyze and deal with signal and information processing in high dimensional space.
High dimensional gaussian classification

Robin Girard

DOI: https://doi.org/10.48550/arXiv.0806.0729

2008-07-10

Abstract:High dimensional data analysis is known to be as a challenging problem. In this article, we give a theoretical analysis of high dimensional classification of Gaussian data which relies on a geometrical analysis of the error measure. It links a problem of classification with a problem of nonparametric regression. We give an algorithm designed for high dimensional data which appears straightforward in the light of our theoretical work, together with the thresholding estimation theory. We finally attempt to give a general treatment of the problem that can be extended to frameworks other than gaussian.

Statistics Theory,Machine Learning
High dimensional statistical inference: theoretical development to data analytics

Deepak Nag Ayyala

DOI: https://doi.org/10.48550/arXiv.1908.06600

2019-08-19

Abstract:This article is due to appear in the Handbook of Statistics, Vol. 43, Elsevier/North-Holland, Amsterdam, edited by Arni S. R. Srinivasa Rao and C. R. Rao. In modern day analytics, there is ever growing need to develop statistical models to study high dimensional data. Between dimension reduction, asymptotics-driven methods and random projection based methods, there are several approaches developed so far. For high dimensional parametric models, estimation and hypothesis testing for mean and covariance matrices have been extensively studied. However, practical implementation of these methods are fairly limited and are primarily restricted to researchers involved in high dimensional inference. With several applied fields such as genomics, metagenomics and social networking, high dimensional inference is a key component of big data analytics. In this chapter, a comprehensive overview of high dimensional inference and its applications in data analytics is provided. Key theoretical developments and computational tools are presented, giving readers an in-depth understanding of challenges in big data analysis.

Statistics Theory,Methodology
Extremes and Records

Sanjib Sabhapandit

DOI: https://doi.org/10.48550/arXiv.1907.00944

2019-07-02

Abstract:These are lecture notes from a course offered at the Bangalore School on Statistical Physics - X, during 17-28 June 2019, [ <a class="link-external link-https" href="https://www.icts.res.in/program/bssp2019" rel="external noopener nofollow">this https URL</a> ] at International centre of theoretical physics (ICTS), Bangalore. These pedagogical lectures are at the introductory level, intended mainly for master/Ph.D. students or researchers from outside the field. In these lectures, we discuss about the limit laws for the sample mean and the maximum of a set of independent and identically distributed (i.i.d.) random variables as well as random walks / Brownian motion. The density of near-extreme events is also discussed. Finally, we discuss the statistics of records for an i.i.d. random sequence as well as random walks in discrete and continuous time. Some exercises are provided for the students to work out. The video recording of the lectures are available at <a class="link-external link-https" href="https://www.icts.res.in/program/bssp2019/talks" rel="external noopener nofollow">this https URL</a>

Statistical Mechanics,Data Analysis, Statistics and Probability
Attractiveness and activity in Internet communities

Gourab Ghoshal,Petter Holme

DOI: https://doi.org/10.1016/j.physa.2005.04.047

2005-04-23

Abstract:Datasets of online communication often take the form of contact sequences -- ordered lists contacts (where a contact is defined as a triple of a sender, a recipient and a time). We propose measures of attractiveness and activity for such data sets and analyze these quantities for anonymized contact sequences from an Internet dating community. For this data set the attractiveness and activity measures show broad power-law like distributions. Our attractiveness and activity measures are more strongly correlated in the real-world data than in our reference model. Effects that indirectly can make active users more attractive are discussed.

Physics and Society
Lectures on random matrix theory and symmetric spaces

Ulrika Magnea

DOI: https://doi.org/10.48550/arXiv.math-ph/0502015

2005-02-04

Mathematical Physics

Abstract:In these lectures we discuss some elementary concepts in connection with the theory of symmetric spaces applied to ensembles of random matrices. We review how the relationship between random matrix theory and symmetric spaces can be used in some physical contexts.
Three lectures on free probability

Jonathan Novak,Michael LaCroix

DOI: https://doi.org/10.48550/arXiv.1205.2097

2012-05-10

Abstract:These are notes from a three-lecture mini-course on free probability given at MSRI in the Fall of 2010 and repeated a year later at Harvard. The lectures were aimed at mathematicians and mathematical physicists working in combinatorics, probability, and random matrix theory. The first lecture was a staged rediscovery of free independence from first principles, the second dealt with the additive calculus of free random variables, and the third focused on random matrix models.

Combinatorics,Mathematical Physics,Probability
Foundations of Descriptive and Inferential Statistics

Henk van Elst

DOI: https://doi.org/10.13140/RG.2.1.2112.3044

2019-08-30

Abstract:These lecture notes were written with the aim to provide an accessible though technically solid introduction to the logic of systematical analyses of statistical data to both undergraduate and postgraduate students, in particular in the Social Sciences, Economics, and the Financial Services. They may also serve as a general reference for the application of quantitative--empirical research methods. In an attempt to encourage the adoption of an interdisciplinary perspective on quantitative problems arising in practice, the notes cover the four broad topics (i) descriptive statistical processing of raw data, (ii) elementary probability theory, (iii) the operationalisation of one-dimensional latent statistical variables according to Likert's widely used scaling approach, and (iv) null hypothesis significance testing within the frequentist approach to probability theory concerning (a) distributional differences of variables between subgroups of a target population, and (b) statistical associations between two variables. The relevance of effect sizes for making inferences is emphasised. These lecture notes are fully hyperlinked, thus providing a direct route to original scientific papers as well as to interesting biographical information. They also list many commands for running statistical functions and data analysis routines in the software packages R, SPSS, EXCEL and OpenOffice. The immediate involvement in actual data analysis practices is strongly recommended.

Applications,Other Statistics
Random matrix theory: Wigner-Dyson statistics and beyond. (Lecture notes of a course given at SISSA (Trieste, Italy))

V.E.Kravtsov

DOI: https://doi.org/10.48550/arXiv.0911.0639

2012-08-24

Abstract:This is a course on Random Matrix Theory which includes traditional as well as advanced topics presented with an extensive use of classical logarithmic plasma analogy and that of the quantum systems of one-dimensional interacting fermions with inverse square interaction (Calogero-Sutherland model). Certain non-invariant random matrix ensembles are also considered with the emphasis on the eigenfunction statistics in them. The course can also be viewed as introduction to theory of localization where the (non-invariant) random matrix ensembles play a role of the toy models to illustrate functional methods based on super-vector/super-matrix representations.

Disordered Systems and Neural Networks,Statistical Mechanics
Kernels, Data & Physics

Francesco Cagnetta,Deborah Oliveira,Mahalakshmi Sabanayagam,Nikolaos Tsilivis,Julia Kempe

2023-07-06

Abstract:Lecture notes from the course given by Professor Julia Kempe at the summer school "Statistical physics of Machine Learning" in Les Houches. The notes discuss the so-called NTK approach to problems in machine learning, which consists of gaining an understanding of generally unsolvable problems by finding a tractable kernel formulation. The notes are mainly focused on practical applications such as data distillation and adversarial robustness, examples of inductive bias are also discussed.

Machine Learning
Lecture Notes in Lie Groups

Vladimir G. Ivancevic,Tijana T. Ivancevic

DOI: https://doi.org/10.48550/arXiv.1104.1106

2011-04-06

Differential Geometry

Abstract:These lecture notes in Lie Groups are designed for a 1--semester third year or graduate course in mathematics, physics, engineering, chemistry or biology. This landmark theory of the 20th Century mathematics and physics gives a rigorous foundation to modern dynamics, as well as field and gauge theories in physics, engineering and biomechanics. We give both physical and medical examples of Lie groups. The only necessary background for comprehensive reading of these notes are advanced calculus and linear algebra.
Extrema of the two-dimensional Discrete Gaussian Free Field

Marek Biskup

DOI: https://doi.org/10.1007/978-3-030-32011-9

2019-08-17

Abstract:These lecture notes offer a gentle introduction to the two-dimensional Discrete Gaussian Free Field with particular attention paid to the scaling limits of the level sets at heights proportional to the absolute maximum. The bulk of the text is based on recent joint papers with O. Louidor and with J. Ding and S. Goswami. Still, new proofs of the tightness and distributional convergence of the centered DGFF maximum are presented that by-pass the use of the modified Branching Random Walk. The text contains a wealth of instructive exercises and a list of open questions and conjectures for future research.

Probability,Mathematical Physics

Lecture notes on high-dimensional data

Lecture Notes on "Random Matrices"

High-dimensional non-convex landscapes and gradient descent dynamics

Mathematics of Neural Networks (Lecture Notes Graduate Course)

Lecture Notes on Free Probability

High-Dimensional Data Analysis with Low-Dimensional Models

Lecture notes on the Fundamental Structures of Differential Geometry

Algorithms for Massive Data -- Lecture Notes

Lecture notes on signal processing in high dimensional space

High dimensional gaussian classification

High dimensional statistical inference: theoretical development to data analytics

Extremes and Records

Attractiveness and activity in Internet communities

Lectures on random matrix theory and symmetric spaces

Three lectures on free probability

Foundations of Descriptive and Inferential Statistics

Random matrix theory: Wigner-Dyson statistics and beyond. (Lecture notes of a course given at SISSA (Trieste, Italy))

Kernels, Data & Physics

Lecture Notes in Lie Groups

Extrema of the two-dimensional Discrete Gaussian Free Field