Abstract:Observations from dynamical systems often exhibit irregularities, such as censoring, where values are recorded only if they fall within a certain range. Censoring is ubiquitous in practice, due to saturating sensors, limit-of-detection effects, and image-frame effects. In light of recent developments on learning linear dynamical systems (LDSs), and on censored statistics with independent data, we revisit the decades-old problem of learning an LDS, from censored observations (Lee and Maddala (1985); Zeger and Brookmeyer (1986)). Here, the learner observes the state $x_t \in \mathbb{R}^d$ if and only if $x_t$ belongs to some set $S_t \subseteq \mathbb{R}^d$. We develop the first computationally and statistically efficient algorithm for learning the system, assuming only oracle access to the sets $S_t$. Our algorithm, Stochastic Online Newton with Switching Gradients, is a novel second-order method that builds on the Online Newton Step (ONS) of Hazan et al. (2007). Our Switching-Gradient scheme does not always use (stochastic) gradients of the function we want to optimize, which we call "censor-aware" function. Instead, in each iteration, it performs a simple test to decide whether to use the censor-aware, or another "censor-oblivious" function, for getting a stochastic gradient. In our analysis, we consider a "generic" Online Newton method, which uses arbitrary vectors instead of gradients, and we prove an error-bound for it. This can be used to appropriately design these vectors, leading to our Switching-Gradient scheme. This framework significantly deviates from the recent long line of works on censored statistics (e.g., Daskalakis et al. (2018); Kontonis et al. (2019); Daskalakis et al. (2019)), which apply Stochastic Gradient Descent (SGD), and their analysis reduces to establishing conditions for off-the-shelf SGD-bounds.

Decentralized RLS with Data-Adaptive Censoring for Regressions over Large-Scale Networks

Distributed Censored Regression over Networks

Distributed Sparse Recursive Least-Squares over Networks

Distributed Censored Regression Over Networks.

CMDRL: A Markovian Distributed Rate Limiting Algorithm in Cloud Networks

Low-complexity Correlated Time-Averaged Variable Forgetting Factor Mechanism for Diffusion RLS Algorithm in Sensor Networks

Censored Regression with Noisy Input

One-Bit Recursive Least-Squares Algorithm with Application to Distributed Target Localization

Covert and Reliable Semantic Communication Against Cross-Layer Privacy Inference over Wireless Edge Networks

Distributed Sparse Total Least-Squares over Networks

Finite-Time Convergence Rates of Decentralized Local Markovian Stochastic Approximation

Decentralized Federated Learning: Balancing Communication and Computing Costs

Asymptotically Efficient Online Learning for Censored Regression Models Under Non-I.I.D Data

Conformal Distributed Remote Inference in Sensor Networks Under Reliability and Communication Constraints

Communication-Censored Distributed Stochastic Gradient Descent

Robust DCD-Based Recursive Adaptive Algorithms

Learning from Censored and Dependent Data: The case of Linear Dynamics

Analysis and Optimization of Sparse Random Linear Network Coding for Reliable Multicast Services

A New Penalized Recursive Least Squares Method With a Variable Regularization Factor for Adaptive Sparse Filtering.

On the Price of Decentralization in Decentralized Detection

Distributed Recursive Filtering Over Sensor Networks Under Random Access Protocol: When State Saturation Meets Censored Measurement