Stereoselective synthesis of spirocyclic oxindoles via Prins cyclizations.

M. Castaldi,D. Troast,J. Porco

DOI: https://doi.org/10.1021/ol901201k

IF: 5.2

2009-08-06

Organic Letters

Abstract:The synthesis of spirocyclic oxindole pyran and oxepene frameworks using highly stereoselective Prins cyclizations of homoallylic and bis-homoallylic alcohols and isatin ketals is described.

What problem does this paper attempt to address?

Nonparametric Operator-Regularized Covariance Function Estimation for Functional Data

Raymond K. W. Wong,Xiaoke Zhang

DOI: https://doi.org/10.48550/arXiv.1701.06263

2017-01-23

Abstract:In functional data analysis (FDA), covariance function is fundamental not only as a critical quantity for understanding elementary aspects of functional data but also as an indispensable ingredient for many advanced FDA methods. This paper develops a new class of nonparametric covariance function estimators in terms of various spectral regularizations of an operator associated with a reproducing kernel Hilbert space. Despite their nonparametric nature, the covariance estimators are automatically positive semi-definite without any additional modification steps. An unconventional representer theorem is established to provide a finite dimensional representation for this class of covariance estimators, which leads to a closed-form expression of the corresponding $L^2$ eigen-decomposition. Trace-norm regularization is particularly studied to further achieve a low-rank representation, another desirable property which leads to dimension reduction and is often needed in advanced FDA approaches. An efficient algorithm is developed based on the accelerated proximal gradient method. This resulted estimator is shown to enjoy an excellent rate of convergence under both fixed and random designs. The outstanding practical performance of the trace-norm-regularized covariance estimator is demonstrated by a simulation study and the analysis of a traffic dataset.

Methodology
Adaptive estimation of irregular mean and covariance functions

Steven Golovkine,Nicolas Klutchnikoff,Valentin Patilea

2024-06-10

Abstract:Nonparametric estimators for the mean and the covariance functions of functional data are proposed. The setup covers a wide range of practical situations. The random trajectories are, not necessarily differentiable, have unknown regularity, and are measured with error at discrete design points. The measurement error could be heteroscedastic. The design points could be either randomly drawn or common for all curves. The estimators depend on the local regularity of the stochastic process generating the functional data. We consider a simple estimator of this local regularity which exploits the replication and regularization features of functional data. Next, we use the ``smoothing first, then estimate'' approach for the mean and the covariance functions. They can be applied with both sparsely or densely sampled curves, are easy to calculate and to update, and perform well in simulations. Simulations built upon an example of real data set, illustrate the effectiveness of the new approach.

Statistics Theory
Covariance Function Estimation for High-Dimensional Functional Time Series with Dual Factor Structures

Chenlei Leng,Degui Li,Hanlin Shang,Yingcun Xia

DOI: https://doi.org/10.48550/arXiv.2401.05784

2024-01-12

Abstract:We propose a flexible dual functional factor model for modelling high-dimensional functional time series. In this model, a high-dimensional fully functional factor parametrisation is imposed on the observed functional processes, whereas a low-dimensional version (via series approximation) is assumed for the latent functional factors. We extend the classic principal component analysis technique for the estimation of a low-rank structure to the estimation of a large covariance matrix of random functions that satisfies a notion of (approximate) functional "low-rank plus sparse" structure; and generalise the matrix shrinkage method to functional shrinkage in order to estimate the sparse structure of functional idiosyncratic components. Under appropriate regularity conditions, we derive the large sample theory of the developed estimators, including the consistency of the estimated factors and functional factor loadings and the convergence rates of the estimated matrices of covariance functions measured by various (functional) matrix norms. Consistent selection of the number of factors and a data-driven rule to choose the shrinkage parameter are discussed. Simulation and empirical studies are provided to demonstrate the finite-sample performance of the developed model and estimation methodology.

Econometrics,Methodology
Adaptive functional principal components analysis

Sunny G.W. Wang,Valentin Patilea,Nicolas Klutchnikoff

2024-04-09

Abstract:Functional data analysis almost always involves smoothing discrete observations into curves, because they are never observed in continuous time and rarely without error. Although smoothing parameters affect the subsequent inference, data-driven methods for selecting these parameters are not well-developed, frustrated by the difficulty of using all the information shared by curves while being computationally efficient. On the one hand, smoothing individual curves in an isolated, albeit sophisticated way, ignores useful signals present in other curves. On the other hand, bandwidth selection by automatic procedures such as cross-validation after pooling all the curves together quickly become computationally unfeasible due to the large number of data points. In this paper we propose a new data-driven, adaptive kernel smoothing, specifically tailored for functional principal components analysis through the derivation of sharp, explicit risk bounds for the eigen-elements. The minimization of these quadratic risk bounds provide refined, yet computationally efficient bandwidth rules for each eigen-element separately. Both common and independent design cases are allowed. Rates of convergence for the estimators are derived. An extensive simulation study, designed in a versatile manner to closely mimic the characteristics of real data sets supports our methodological contribution. An illustration on a real data application is provided.

Statistics Theory
Statistical inferences for functional data

Jin-Ting Zhang,Jianwei Chen

DOI: https://doi.org/10.48550/arXiv.0708.2207

2007-08-16

Statistics Theory

Abstract:With modern technology development, functional data are being observed frequently in many scientific fields. A popular method for analyzing such functional data is ``smoothing first, then estimation.'' That is, statistical inference such as estimation and hypothesis testing about functional data is conducted based on the substitution of the underlying individual functions by their reconstructions obtained by one smoothing technique or another. However, little is known about this substitution effect on functional data analysis. In this paper this problem is investigated when the local polynomial kernel (LPK) smoothing technique is used for individual function reconstructions. We find that under some mild conditions, the substitution effect can be ignored asymptotically. Based on this, we construct LPK reconstruction-based estimators for the mean, covariance and noise variance functions of a functional data set and derive their asymptotics. We also propose a GCV rule for selecting good bandwidths for the LPK reconstructions. When the mean function also depends on some time-independent covariates, we consider a functional linear model where the mean function is linearly related to the covariates but the covariate effects are functions of time. The LPK reconstruction-based estimators for the covariate effects and the covariance function are also constructed and their asymptotics are derived. Moreover, we propose a $L^2$-norm-based global test statistic for a general hypothesis testing problem about the covariate effects and derive its asymptotic random expression. The effect of the bandwidths selected by the proposed GCV rule on the accuracy of the LPK reconstructions and the mean function estimator is investigated via a simulation study. The proposed methodologies are illustrated via an application to a real functional data set collected in climatology.
Functional PCA With Covariate-Dependent Mean and Covariance Structure

Fei Ding,Shiyuan He,David E. Jones,Jianhua Z. Huang

DOI: https://doi.org/10.1080/00401706.2021.2008502

2022-01-04

Technometrics

Abstract:Incorporating covariates into functional principal component analysis (PCA) can substantially improve the representation efficiency of the principal components and predictive performance. However, many existing functional PCA methods do not make use of covariates, and those that do often have high computational cost or make overly simplistic assumptions that are violated in practice. In this article, we propose a new framework, called covariate-dependent functional principal component analysis (CD-FPCA), in which both the mean and covariance structure depend on covariates. We propose a corresponding estimation algorithm, which makes use of spline basis representations and roughness penalties, and is substantially more computationally efficient than competing approaches of adequate estimation and prediction accuracy. A key aspect of our work is our novel approach for modeling the covariance function and ensuring that it is symmetric positive semidefinite. We demonstrate the advantages of our methodology through a simulation study and an astronomical data analysis.

statistics & probability
Fast covariance estimation for multivariate sparse functional data

Cai Li,Luo Xiao,Sheng Luo

DOI: https://doi.org/10.1002/sta4.245

2020-01-01

Stat

Abstract:<p>Covariance estimation is essential yet underdeveloped for analysing multivariate functional data. We propose a fast covariance estimation method for multivariate sparse functional data using bivariate penalized splines. The tensor‐product B‐spline formulation of the proposed method enables a simple spectral decomposition of the associated covariance operator and explicit expressions of the resulting eigenfunctions as linear combinations of B‐spline bases, thereby dramatically facilitating subsequent principal component analysis. We derive a fast algorithm for selecting the smoothing parameters in covariance smoothing using leave‐one‐subject‐out cross‐validation. The method is evaluated with extensive numerical studies and applied to an Alzheimer's disease study with multiple longitudinal outcomes.</p>

statistics & probability
Mean and Covariance Estimation for Discretely Observed High-Dimensional Functional Data: Rates of Convergence and Division of Observational Regimes

Alexander Petersen

2024-08-05

Abstract:Estimation of the mean and covariance parameters for functional data is a critical task, with local linear smoothing being a popular choice. In recent years, many scientific domains are producing multivariate functional data for which $p$, the number of curves per subject, is often much larger than the sample size $n$. In this setting of high-dimensional functional data, much of developed methodology relies on preliminary estimates of the unknown mean functions and the auto- and cross-covariance functions. This paper investigates the convergence rates of local linear estimators in terms of the maximal error across components and pairs of components for mean and covariance functions, respectively, in both $L^2$ and uniform metrics. The local linear estimators utilize a generic weighting scheme that can adjust for differing numbers of discrete observations $N_{ij}$ across curves $j$ and subjects $i$, where the $N_{ij}$ vary with $n$. Particular attention is given to the equal weight per observation (OBS) and equal weight per subject (SUBJ) weighting schemes. The theoretical results utilize novel applications of concentration inequalities for functional data and demonstrate that, similar to univariate functional data, the order of the $N_{ij}$ relative to $p$ and $n$ divides high-dimensional functional data into three regimes (sparse, dense, and ultra-dense), with the high-dimensional parametric convergence rate of $\left\{\log(p)/n\right\}^{1/2}$ being attainable in the latter two.

Statistics Theory,Methodology
Separable Expansions for Covariance Estimation

Tomas Masak,Soham Sarkar,Victor M. Panaretos

DOI: https://doi.org/10.48550/arXiv.2007.12175

2022-01-18

Abstract:The non-parametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are typically alleviated by assuming separability. However, separability is often questionable, sometimes even demonstrably inadequate. We propose a framework for the analysis of covariance operators of random surfaces that generalises separability, while retaining its major advantages. Our approach is based on the expansion of the covariance into a series of separable terms. The expansion is valid for any covariance over a two-dimensional domain. Leveraging the key notion of the partial inner product, we extend the power iteration method to general Hilbert spaces and show how the aforementioned expansion can be efficiently constructed in practice. Truncation of the expansion and retention of the leading terms automatically induces a non-parametric estimator of the covariance, whose parsimony is dictated by the truncation level. The resulting estimator can be calculated, stored and manipulated with little computational overhead relative to separability. Consistency and rates of convergence are derived under mild regularity assumptions, illustrating the trade-off between bias and variance regulated by the truncation level. The merits and practical performance of the proposed methodology are demonstrated in a comprehensive simulation study and on classification of EEG signals.

Statistics Theory,Computation,Methodology
Oracle-Efficient Confidence Envelopes for Covariance Functions in Dense Functional Data

Guanqun Cao,Li Wang,Yehua Li,Lijian Yang

DOI: https://doi.org/10.5705/ss.2014.182

IF: 1.4

2016-01-01

Statistica Sinica

Abstract:We consider nonparametric estimation of the covariance function for dense functional data using computationally efficient tensor product B-splines. We develop both local and global asymptotic distributions for the proposed estimator, and show that our estimator is as efficient as an "oracle" estimator where the true mean function is known. Simultaneous confidence envelopes are developed based on asymptotic theory to quantify the variability in the covariance estimator and to make global inferences on the true covariance. Monte Carlo simulation experiments provide strong evidence that corroborates the asymptotic theory. Examples of near infrared spectroscopy data and speech recognition data are provided to illustrate the proposed method.
On estimation in varying coefficient models for sparse and irregularly sampled functional data

Behdad Mostafaiy

DOI: https://doi.org/10.1080/03610918.2019.1661478

2019-09-04

Abstract:In this paper, we study a smoothness regularization method for a varying coefficient model based on sparse and irregularly sampled functional data which is contaminated with some measurement errors. We estimate the one-dimensional covariance and cross-covariance functions of the underlying stochastic processes based on a reproducing kernel Hilbert space approach. We then obtain least squares estimates of the coefficient functions. Simulation studies demonstrate that the proposed method has good performance. We illustrate our method by an analysis of longitudinal primary biliary liver cirrhosis data.
Adaptive Functional Thresholding for Sparse Covariance Function Estimation in High Dimensions

Qin Fang,Shaojun Guo,Xinghao Qiao

DOI: https://doi.org/10.1080/01621459.2023.2200522

IF: 4.369

2023-04-06

Journal of the American Statistical Association

Abstract:Covariance function estimation is a fundamental task in multivariate functional data analysis and arises in many applications. In this article, we consider estimating sparse covariance functions for high-dimensional functional data, where the number of random functions p is comparable to, or even larger than the sample size n. Aided by the Hilbert–Schmidt norm of functions, we introduce a new class of functional thresholding operators that combine functional versions of thresholding and shrinkage, and propose the adaptive functional thresholding estimator by incorporating the variance effects of individual entries of the sample covariance function into functional thresholding. To handle the practical scenario where curves are partially observed with errors, we also develop a nonparametric smoothing approach to obtain the smoothed adaptive functional thresholding estimator and its binned implementation to accelerate the computation. We investigate the theoretical properties of our proposals when p grows exponentially with n under both fully and partially observed functional scenarios. Finally, we demonstrate that the proposed adaptive functional thresholding estimators significantly outperform the competitors through extensive simulations and the functional connectivity analysis of two neuroimaging datasets. Supplementary materials for this article are available online.

statistics & probability
Fast Covariance Estimation for Sparse Functional Data

Luo Xiao,Cai Li,William Checkley,Ciprian M. Crainiceanu

DOI: https://doi.org/10.48550/arXiv.1603.05758

2017-04-06

Abstract:Smoothing of noisy sample covariances is an important component in functional data analysis. We propose a novel covariance smoothing method based on penalized splines and associated software. The proposed method is a bivariate spline smoother that is designed for covariance smoothing and can be used for sparse functional or longitudinal data. We propose a fast algorithm for covariance smoothing using leave-one-subject-out cross validation. Our simulations show that the proposed method compares favorably against several commonly used methods. The method is applied to a study of child growth led by one of coauthors and to a public dataset of longitudinal CD4 counts.

Methodology
√2-Estimation for Smooth Eigenvectors of Matrix-Valued Functions

Giovanni Motta,Wei Biao Wu,Mohsen Pourahmadi

DOI: https://doi.org/10.1093/biomet/asad018

IF: 3.0279

2023-03-15

Biometrika

Abstract:Summary Modern statistical methods for multivariate time series rely on the eigendecomposition of matrix-valued functions such as time-varying covariance and spectral density matrices. The curse of indeterminacy or misidentification of smooth eigenvector functions has not received much attention. We resolve this important problem and recover smooth trajectories by examining the distance between the eigenvectors of the same matrix-valued function evaluated at two consecutive points. We change the sign of the next eigenvector if its distance with the current one is larger than the square root of 2. In the case of distinct eigenvalues, this simple method delivers smooth eigenvectors. For coalescing eigenvalues, we match the corresponding eigenvectors and apply an additional signing around the coalescing points. We establish consistency and rates of convergence for the proposed smooth eigenvector estimators. Simulation results and applications to real data confirm that our approach is needed to obtain smooth eigenvectors.

statistics & probability,mathematical & computational biology,biology
Theory of functional principal component analysis for discretely observed data

Hang Zhou,Dongyi Wei,Fang Yao

2024-04-02

Abstract:Functional data analysis is an important research field in statistics which treats data as random functions drawn from some infinite-dimensional functional space, and functional principal component analysis (FPCA) based on eigen-decomposition plays a central role for data reduction and representation. After nearly three decades of research, there remains a key problem unsolved, namely, the perturbation analysis of covariance operator for diverging number of eigencomponents obtained from noisy and discretely observed data. This is fundamental for studying models and methods based on FPCA, while there has not been substantial progress since Hall, Müller and Wang (2006)'s result for a fixed number of eigenfunction estimates. In this work, we aim to establish a unified theory for this problem, obtaining upper bounds for eigenfunctions with diverging indices in both the $\mathcal{L}^2$ and supremum norms, and deriving the asymptotic distributions of eigenvalues for a wide range of sampling schemes. Our results provide insight into the phenomenon when the $\mathcal{L}^{2}$ bound of eigenfunction estimates with diverging indices is minimax optimal as if the curves are fully observed, and reveal the transition of convergence rates from nonparametric to parametric regimes in connection to sparse or dense sampling. We also develop a double truncation technique to handle the uniform convergence of estimated covariance and eigenfunctions. The technical arguments in this work are useful for handling the perturbation series with noisy and discretely observed functional data and can be applied in models or those involving inverse problems based on FPCA as regularization, such as functional linear regression.

Statistics Theory
Factor-guided estimation of large covariance matrix function with conditional functional sparsity

Dong Li,Xinghao Qiao,Zihan Wang

DOI: https://doi.org/10.48550/arXiv.2311.02450

2023-11-04

Methodology

Abstract:This paper addresses the fundamental task of estimating covariance matrix functions for high-dimensional functional data/functional time series. We consider two functional factor structures encompassing either functional factors with scalar loadings or scalar factors with functional loadings, and postulate functional sparsity on the covariance of idiosyncratic errors after taking out the common unobserved factors. To facilitate estimation, we rely on the spiked matrix model and its functional generalization, and derive some novel asymptotic identifiability results, based on which we develop DIGIT and FPOET estimators under two functional factor models, respectively. Both estimators involve performing associated eigenanalysis to estimate the covariance of common components, followed by adaptive functional thresholding applied to the residual covariance. We also develop functional information criteria for the purpose of model selection. The convergence rates of estimated factors, loadings, and conditional sparse covariance matrix functions under various functional matrix norms, are respectively established for DIGIT and FPOET estimators. Numerical studies including extensive simulations and two real data applications on mortality rates and functional portfolio allocation are conducted to examine the finite-sample performance of the proposed methodology.
Online Estimation for Functional Data

Ying Yang,Fang Yao

DOI: https://doi.org/10.1080/01621459.2021.2002158

IF: 4.369

2021-01-01

Journal of the American Statistical Association

Abstract:Functional data analysis has attracted considerable interest and is facing new challenges, one of which is the increasingly available data in a streaming manner. In this article we develop an online nonparametric method to dynamically update the estimates of mean and covariance functions for functional data. The kernel-type estimates can be decomposed into two sufficient statistics depending on the data-driven bandwidths. We propose to approximate the future optimal bandwidths by a sequence of dynamically changing candidates and combine the corresponding statistics across blocks to form the updated estimation. The proposed online method is easy to compute based on the stored sufficient statistics and the current data block. We derive the asymptotic normality and, more importantly, the relative efficiency lower bounds of the online estimates of mean and covariance functions. This provides insight into the relationship between estimation accuracy and computational cost driven by the length of candidate bandwidth sequence. Simulations and real data examples are provided to support such findings. Supplementary materials for this article are available online.
Simultaneous Confidence Band for Stationary Covariance Function of Dense Functional Data

Wang Jiangyan,Cao Guanqun,Wang Li,Yang Lijian

DOI: https://doi.org/10.1016/j.jmva.2019.104584

IF: 1.387

2019-01-01

Journal of Multivariate Analysis

Abstract:The inference via simultaneous confidence band is studied for stationary covariance function of dense functional data. A two-stage estimation procedure is proposed based on spline approximation, the first stage involving estimation of all the individual trajectories and the second stage involving estimation of the covariance function through smoothing the empirical covariance function. The proposed covariance estimator is smooth and as efficient as the oracle estimator when all individual trajectories are known. An asymptotic simultaneous confidence band (SCB) is developed for the true covariance function, and the coverage probabilities are shown to be asymptotically correct. Intensive simulation experiments are conducted to demonstrate the performance of the proposed estimator and SCB. The proposed method is also illustrated with a real data example.
PCA-based estimation for functional linear regression with functional responses

Masaaki Imaizumi,Kengo Kato

DOI: https://doi.org/10.1016/j.jmva.2017.10.001

IF: 1.387

2018-01-01

Journal of Multivariate Analysis

Abstract:This paper studies a regression model where both predictor and response variables are random functions. We consider a functional linear model where the conditional mean of the response variable at each time point is given by a linear functional of the predictor variable. In this paper, we are interested in estimation of the integral kernel b(s,t) of the conditional expectation operator, where s is an output variable while t is a variable that interacts with the predictor variable. This problem is an ill-posed inverse problem, and we consider two estimators based on functional principal component analysis (PCA). We show that under suitable regularity conditions, an estimator based on the single truncation attains the convergence rate for the integrated squared error that is characterized by smoothness of the function b(s,t) in t together with the decay rate of the eigenvalues of the covariance operator, but the rate does not depend on the smoothness of b(s,t) in s. This rate is shown to be minimax optimal, and consequently smoothness of b(s,t) in s does not affect difficulty of estimating b. We also consider an alternative estimator based on the double truncation, and provide conditions under which the alternative estimator attains the optimal rate. We conduct simulations to verify the performance of PCA-based estimators in the finite sample. Finally, we apply our estimators to investigate the relation between the lifetime pattern of working hours and total income, and the relation between the electricity spot price and the wind power in-feed.

statistics & probability

Stereoselective synthesis of spirocyclic oxindoles via Prins cyclizations.

Nonparametric Operator-Regularized Covariance Function Estimation for Functional Data

Adaptive estimation of irregular mean and covariance functions

Covariance Function Estimation for High-Dimensional Functional Time Series with Dual Factor Structures

Adaptive functional principal components analysis

Statistical inferences for functional data

Functional PCA With Covariate-Dependent Mean and Covariance Structure

Fast covariance estimation for multivariate sparse functional data

Mean and Covariance Estimation for Discretely Observed High-Dimensional Functional Data: Rates of Convergence and Division of Observational Regimes

Separable Expansions for Covariance Estimation

Oracle-Efficient Confidence Envelopes for Covariance Functions in Dense Functional Data

On estimation in varying coefficient models for sparse and irregularly sampled functional data

Adaptive Functional Thresholding for Sparse Covariance Function Estimation in High Dimensions

Fast Covariance Estimation for Sparse Functional Data

√2-Estimation for Smooth Eigenvectors of Matrix-Valued Functions

Theory of functional principal component analysis for discretely observed data

Factor-guided estimation of large covariance matrix function with conditional functional sparsity

Online Estimation for Functional Data

Simultaneous Confidence Band for Stationary Covariance Function of Dense Functional Data

PCA-based estimation for functional linear regression with functional responses