The Optimal Stopping Time for Detecting Changes in Discrete Time Markov Processes

D. Han,F. Tsung

DOI: https://doi.org/10.1080/07474940802619451

2009-01-01

Sequential Analysis

Abstract:Abstract In this paper we prove that Page's stopping time (CUSUM test) is optimal for detecting the changes in Markov processes that have invariant distribution.

What problem does this paper attempt to address?

Change Point Detection for Nonparametric Regression under Strongly Mixing Process

Yang Qing,Li Yu-Ning,Zhang Yi

DOI: https://doi.org/10.1007/s00362-020-01196-y

2020-01-01

Statistical Papers

Abstract:In this article, we consider the estimation of the structural change point in the nonparametric model with dependent observations. We introduce a maximum-CUSUM-estimation procedure, where the CUSUM statistic is constructed based on the sum-of-squares aggregation of the difference of the two Nadaraya-Watson estimates using the observations before and after a specific time point. Under some mild conditions, we prove that the statistic tends to zero almost surely if there is no change, and is larger than a threshold asymptotically almost surely otherwise, which helps us to obtain a threshold-detection strategy. Furthermore, we demonstrate the strong consistency of the change point estimator. In the simulation, we discuss the selection of the bandwidth and the threshold used in the estimation, and show the robustness of our method in the long-memory scenario. We implement our method to the data of Nasdaq 100 index and find that the relation between the realized volatility and the return exhibits several structural changes in 2007–2009.
Online Change Detection of Markov Chains with Unknown Post-Change Transition Probabilities

Jin-Guo Xian,Dong Han,Jian-Qi Yu

DOI: https://doi.org/10.1080/03610926.2013.833243

2013-01-01

Communication in Statistics- Theory and Methods

Abstract:In this paper, we investigate the performance of cumulative sum (CUSUM) stopping rules for the online detection of unknown change point in a time homogeneous Markov chain. Under the condition that the post-change transition probabilities are unknown, we proposed two CUSUM type schemes for the detection. The first scheme is based on the maximum likelihood estimates of the post-change transition probabilities. This scheme is limited by its computation burden, which is mitigated by another scheme based on the reference transition probabilities selected from a prior known region. We give the bounds of the mean delay time and the mean time between false alarms to illustrate the effectiveness of the proposed schemes. The results of the simulation also demonstrate the feasibility of the proposed schemes.
Quickest Detection of a State-Dependent Change-Point in Discrete Time

Liang Cai,Li-Yun Pan,Huan-Huan Zhang,Ming-Zhong Gao,Zhi-Nan Wang

DOI: https://doi.org/10.1080/07474946.2017.1394723

2017-01-01

Sequential Analysis

Abstract:We study the quickest detection of a change point in a sequence of observations whose prior probability depends on the observed states. By the Bayesian scheme we get the optimal stopping, rules under the hybrid probability space.
Page's Sequential Procedure for Change-Point Detection in Time Series Regression

Stefan Fremdt

DOI: https://doi.org/10.48550/arXiv.1308.1237

2013-08-06

Abstract:In a variety of different settings cumulative sum (CUSUM) procedures have been applied for the sequential detection of structural breaks in the parameters of stochastic models. Yet their performance depends strongly on the time of change and is best under early-change scenarios. For later changes their finite sample behavior is rather questionable. We therefore propose modified CUSUM procedures for the detection of abrupt changes in the regression parameter of multiple time series regression models, that show a higher stability with respect to the time of change than ordinary CUSUM procedures. The asymptotic distributions of the test statistics and the consistency of the procedures are provided. In a simulation study it is shown that the proposed procedures behave well in finite samples. Finally the procedures are applied to a set of capital asset pricing data related to the Fama-French extension of the capital asset pricing model.

Methodology
On the Optimality of Bayesian Change-Point Detection

Dong Han,Fugee Tsung,Jinguo Xian

DOI: https://doi.org/10.1214/16-aos1479

2017-01-01

The Annals of Statistics

Abstract:By introducing suitable loss random variables of detection, we obtain optimal tests in terms of the stopping time or alarm time for Bayesian changepoint detection not only for a general prior distribution of change-points but also for observations being a Markov process. Moreover, the optimal (minimal) average detection delay is proved to be equal to 1 for any (possibly large) average run length to false alarm if the number of possible change-points is finite.
Exactly Optimal Quickest Change Detection of Markov Chains

Jason J. Ford,Justin M. Kennedy,Caitlin Tompkins,Jasmin James,Aaron McFadyen

2023-03-24

Abstract:This paper establishes that an exactly optimal rule for Bayesian Quickest Change Detection (QCD) of Markov chains is a threshold test on the no change posterior. We also provide a computationally efficient scalar filter for the no change posterior whose effort is independent of the dimension of the chains. We establish that an (undesirable) weak practical super-martingale phenomenon can be exhibited by the no change posterior when the before and after chains are too close in a relative entropy rate sense. The proposed detector is examined in simulation studies.

Systems and Control,Statistics Theory
Delay time in sequential detection of change

Alexander Aue,Lajos Horváth

DOI: https://doi.org/10.1016/j.spl.2004.01.002

2004-04-01

Abstract:We consider a sequential test procedure which detects possible changes in the mean of observations satisfying a weak invariance principle. Our test statistic is based on weighted CUSUMs of the underlying random variables. In this paper, we study the asymptotic behaviour of the delay time if a change has occurred in the sample after a training period of size m in which the observations stay in control. It turns out that in this situation the limiting distribution of the delay time for m→∞ is normal under a suitable standardization provided the change appeared sufficiently soon after m.

statistics & probability
Optimality of Non-Restarting CUSUM charts

F. Din-Houn Lau,Axel Gandy

DOI: https://doi.org/10.48550/arXiv.1205.1937

2012-05-09

Abstract:We show optimality, in a well-defined sense, using cumulative sum (CUSUM) charts for detecting changes in distributions. We consider a setting with multiple changes between two known distributions. This result advocates the use of non-restarting CUSUM charts with an upper boundary. Typically, after signalling, a CUSUM chart is restarted by setting it to some value below the threshold. A non-restarting CUSUM chart is not reset after signalling; thus is able to signal continuously. Imposing an upper boundary prevents the CUSUM chart rising too high, which facilitates detection in our setting. We discuss, via simulations, how the choice of the upper boundary changes the signals made by the non-restarting CUSUM charts.

Methodology
The Optimal CUSUM Control Chart with a Dynamic Non-Random Control Limit and a Given Sampling Strategy for Small Samples Sequence.

Dong Han,Fugee Tsung,Lei Qiao

DOI: https://doi.org/10.1080/02664763.2022.2162863

IF: 1.416

2023-01-01

Journal of Applied Statistics

Abstract:This article proposes a performance measure to evaluate the detection performance of a control chart with a given sampling strategy for finite or small samples sequence and prove that the CUSUM control chart with dynamic non-random control limit and a given sampling strategy can be optimal under the measure. Numerical simulations and real data for an earthquake are provided to illustrate that for different sampling strategies, the CUSUM chart will have different monitoring performance in change-point detection. Among the six sampling strategies that take only a part of samples, the numerical comparing results illustrate that the uniform sampling strategy (uniformly dispersed sampling strategy) has the best monitoring effect.
High Probability Latency Sequential Change Detection over an Unknown Finite Horizon

Yu-Han Huang,Venugopal V. Veeravalli

2024-09-19

Abstract:A finite horizon variant of the quickest change detection problem is studied, in which the goal is to minimize a delay threshold (latency), under constraints on the probability of false alarm and the probability that the latency is exceeded. In addition, the horizon is not known to the change detector. A variant of the cumulative sum (CuSum) test with a threshold that increasing logarithmically with time is proposed as a candidate solution to the problem. An information-theoretic lower bound on the minimum value of the latency under the constraints is then developed. This lower bound is used to establish certain asymptotic optimality properties of the proposed test in terms of the horizon and the false alarm probability. Some experimental results are given to illustrate the performance of the test.

Data Structures and Algorithms,Systems and Control,Statistics Theory
Optimum Multi-Stream Sequential Change-Point Detection With Sampling Control

Qunzhi Xu,Yajun Mei,George V. Moustakides

DOI: https://doi.org/10.1109/tit.2021.3074961

IF: 2.5

2021-11-01

IEEE Transactions on Information Theory

Abstract:In multi-stream sequential change-point detection it is assumed that there are <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="2.442ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 1051.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-4D" x="0" y="0"></use></g></svg></span> processes in a system and at some unknown time, an occurring event changes the distribution of the samples of a particular process. In this article, we consider this problem under a sampling control constraint when one is allowed, at each point in time, to sample a single process. The objective is to raise an alarm as quickly as possible subject to a proper false alarm constraint. We show that under sampling control, a simple myopic-sampling-based sequential change-point detection strategy is second-order asymptotically optimal when the number <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="2.442ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 1051.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-4D" x="0" y="0"></use></g></svg></span> of processes is fixed. This means that the proposed detector, even by sampling with a rate <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="4.767ex" height="2.843ex" style="vertical-align: -0.838ex;" viewBox="0 -863.1 2052.5 1223.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMAIN-31" x="0" y="0"></use> <use xlink:href="#MJMAIN-2F" x="500" y="0"></use> <use xlink:href="#MJMATHI-4D" x="1001" y="0"></use></g></svg></span> of the full rate, enjoys the same detection delay, up to some additive finite constant, as the optimal procedure. Simulation experiments corroborate our theoretical results.<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><defs id="MathJax_SVG_glyphs"><path stroke-width="1" id="MJMATHI-4D" d="M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z"></path><path stroke-width="1" id="MJMAIN-31" d="M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z"></path><path stroke-width="1" id="MJMAIN-2F" d="M423 750Q432 750 438 744T444 730Q444 725 271 248T92 -240Q85 -250 75 -250Q68 -250 62 -245T56 -231Q56 -221 230 257T407 740Q411 750 423 750Z"></path></defs></svg>

computer science, information systems,engineering, electrical & electronic
Numerical solutions of optimal stopping problems for a class of hybrid stochastic systems

Philip A. Ernst,Xiaohang Ma,Masoud H. Nazari,Hongjiang Qian,Le Yi Wang,George Yin

DOI: https://doi.org/10.1016/j.nahs.2024.101507

IF: 5.477

2024-05-18

Nonlinear Analysis Hybrid Systems

Abstract:This paper is devoted to numerically solving a class of optimal stopping problems for stochastic hybrid systems involving both continuous states and discrete events. The motivation for solving this class of problems stems from quickest event detection problems of stochastic hybrid systems in broad application domains. We solve the optimal stopping problems numerically by constructing feasible algorithms using Markov chain approximation techniques. The key tasks we undertake include designing and constructing discrete-time Markov chains that are locally consistent with switching diffusions, proving the convergence of suitably scaled sequences, and obtaining convergence for the cost and value functions. Finally, numerical results are provided to demonstrate the performance of the algorithms.

automation & control systems,mathematics, applied
Quickest Change Detection Using Mismatched CUSUM

Austin Cooper,Sean Meyn

2024-09-12

Abstract:The field of quickest change detection (QCD) concerns design and analysis of algorithms to estimate in real time the time at which an important event takes place and identify properties of the post-change behavior. The goal is to devise a stopping time adapted to the observations that minimizes an $L_1$ loss. Approximately optimal solutions are well known under a variety of assumptions. In the work surveyed here we consider the CUSUM statistic, which is defined as a one-dimensional reflected random walk driven by a functional of the observations. It is known that the optimal functional is a log likelihood ratio subject to special statical assumptions. The paper concerns model free approaches to detection design, considering the following questions: 1. What is the performance for a given functional of the observations? 2. How do the conclusions change when there is dependency between pre- and post-change behavior? 3. How can techniques from statistics and machine learning be adapted to approximate the best functional in a given class?

Statistics Theory,Information Theory
A detection problem with a monotone observation rate

Erik Ekström,Alessandro Milazzo

DOI: https://doi.org/10.1016/j.spa.2024.104337

IF: 1.43

2024-03-14

Stochastic Processes and their Applications

Abstract:We study a quickest detection problem where the observation rate of the underlying process can be increased at any time for higher precision, but at an observation cost that grows linearly in the observation rate. This leads to a problem of combined control-and-stopping with incomplete information , with a two-dimensional sufficient statistic comprised of the current observation rate together with the conditional probability that disorder has already happened. The problem is shown to have a semi-explicit solution, where for some parameter values it is too costly to exert control at all, whereas for other parameter values the optimal strategy is to increase the observation rate in such a way that the sufficient statistic reflects at a certain boundary until the optimal stopping time. In both cases we fully characterise the optimal strategy with the help of appropriate smooth fit conditions.

statistics & probability
An CUSUM Test with Observation-Adjusted Control Limits in Change Detection

Fuquan Tang,Dong Han

2023-01-01

Abstract: In this paper, we not only propose an new optimal sequential test of sum of logarithmic likelihood ratio (SLR) but also present the CUSUM sequential test (control chart, stopping time) with the observation-adjusted control limits (CUSUM-OAL) for monitoring quickly and adaptively the change in distribution of a sequential observations. Two limiting relationships between the optimal test and a series of the CUSUM-OAL tests are established. Moreover, we give the estimation of the in-control and the out-of-control average run lengths (ARLs) of the CUSUM-OAL test. The theoretical results are illustrated by numerical simulations in detecting mean shifts of the observations sequence.
Stochastic Processes with Expected Stopping Time

Krishnendu Chatterjee,Laurent Doyen

2024-09-10

Abstract:Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish inter-reducibility of the expected stopping-time problem for Markov chains with the Positivity problem (which is related to the well-known Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.

Logic in Computer Science
Sequential change point detection in high dimensional time series

Josua Gösmann,Christina Stoehr,Johannes Heiny,Holger Dette

DOI: https://doi.org/10.48550/arXiv.2006.00636

2020-12-15

Abstract:Change point detection in high dimensional data has found considerable interest in recent years. Most of the literature either designs methodology for a retrospective analysis, where the whole sample is already available when the statistical inference begins, or considers online detection schemes controlling the average time until a false alarm. This paper takes a different point of view and develops monitoring schemes for the online scenario, where high dimensional data arrives successively and the goal is to detect changes as fast as possible controlling at the same time the probability of a type I error of a false alarm. We develop a sequential procedure capable of detecting changes in the mean vector of a successively observed high dimensional time series with spatial and temporal dependence. The statistical properties of the method are analyzed in the case where both, thesample size and dimension tend to infinity. In this scenario, it is shown that the new monitoring scheme has asymptotic level alpha under the null hypothesis of no change and is consistent under the alternative of a change in at least one component of the high dimensional mean vector. The approach is based on a new type of monitoring scheme for one-dimensional data which turns out to be often more powerful than the usually used CUSUM and Page-CUSUM methods, and the component-wise statistics are aggregated by the maximum statistic. For the analysis of the asymptotic properties of our monitoring scheme we prove that the range of a Brownian motion on a given interval is in the domain of attraction of the Gumbel distribution, which is a result of independent interest in extreme value theory. The finite sample properties of the new methodology are illustrated by means of a simulation study and in the analysis of a data example.

Statistics Theory
Detection and Diagnosis of Unknown Abrupt Changes Using CUSUM Multi-Chart Schemes

Dong Han,Fugee Tsung

DOI: https://doi.org/10.1080/07474940701404765

2007-01-01

Sequential Analysis

Abstract:Abstract A cumulative sum (CUSUM) multi-chart scheme that consists of multiple CUSUM control charts is studied for detecting and diagnosing an unknown abrupt change in a stochastic system on the basis of sequential observations. We prove that the CUSUM multi-chart not only has a high diagnostic capability but also possesses a better detection performance than individual CUSUM charts when the in-control average run length is large. We also present an optimal design of the CUSUM multi-chart and two illustrative examples involving the normal and exponential distributions. Moreover, numerical comparisons of the average run lengths are made via Monte Carlo simulation among the CUSUM, generalized likelihood ratio, exponentially weighted moving average (EWMA), multi-chart, and CUSUM multi-chart. The numerical results indicate that the CUSUM multi-chart has the best performance on the whole among the five schemes in detecting the unknown mean shift.
Data-driven optimal stopping: A pure exploration analysis

Sören Christensen,Niklas Dexheimer,Claudia Strauch

2023-12-10

Abstract:The standard theory of optimal stopping is based on the idealised assumption that the underlying process is essentially known. In this paper, we drop this restriction and study data-driven optimal stopping for a general diffusion process, focusing on investigating the statistical performance of the proposed estimator of the optimal stopping barrier. More specifically, we derive non-asymptotic upper bounds on the simple regret, along with uniform and non-asymptotic PAC bounds. Minimax optimality is verified by completing the upper bound results with matching lower bounds on the simple regret. All results are shown both under general conditions on the payoff functions and under more refined assumptions that mimic the margin condition used in binary classification, leading to an improved rate of convergence. Additionally, we investigate how our results on the simple regret transfer to the cumulative regret for a specific exploration-exploitation strategy, both with respect to lower bounds and upper bounds.

Statistics Theory,Optimization and Control,Machine Learning
Optimal Change-point Testing for High-dimensional Linear Models with Temporal Dependence

Zifeng Zhao,Xiaokai Luo,Zongge Liu,Daren Wang

2024-10-22

Abstract:In this paper, we study change-point testing for high-dimensional linear models, an important problem that has not been well explored in the literature. Specifically, we propose a quadratic-form cumulative sum (CUSUM) statistic to test the stability of regression coefficients in high-dimensional linear models. The test controls type-I error at any desired level and is robust to temporally dependent observations. We establish its asymptotic distribution under the null hypothesis, and demonstrate that it is asymptotically powerful against multiple change-point alternatives and achieves the optimal detection boundary for a wide class of high-dimensional models. We further develop an adaptive procedure to estimate the tuning parameters of the test, making our method practical in applications. Additionally, we extend our approach to localize change-points in the regression time series and establish sharp error bounds for our change-point estimator. Extensive numerical experiments and a real data application in macroeconomics are conducted to demonstrate the promising performance and practical utility of the proposed test.

Statistics Theory

The Optimal Stopping Time for Detecting Changes in Discrete Time Markov Processes

Change Point Detection for Nonparametric Regression under Strongly Mixing Process

Online Change Detection of Markov Chains with Unknown Post-Change Transition Probabilities

Quickest Detection of a State-Dependent Change-Point in Discrete Time

Page's Sequential Procedure for Change-Point Detection in Time Series Regression

On the Optimality of Bayesian Change-Point Detection

Exactly Optimal Quickest Change Detection of Markov Chains

Delay time in sequential detection of change

Optimality of Non-Restarting CUSUM charts

The Optimal CUSUM Control Chart with a Dynamic Non-Random Control Limit and a Given Sampling Strategy for Small Samples Sequence.

High Probability Latency Sequential Change Detection over an Unknown Finite Horizon

Optimum Multi-Stream Sequential Change-Point Detection With Sampling Control

Numerical solutions of optimal stopping problems for a class of hybrid stochastic systems

Quickest Change Detection Using Mismatched CUSUM

A detection problem with a monotone observation rate

An CUSUM Test with Observation-Adjusted Control Limits in Change Detection

Stochastic Processes with Expected Stopping Time

Sequential change point detection in high dimensional time series

Detection and Diagnosis of Unknown Abrupt Changes Using CUSUM Multi-Chart Schemes

Data-driven optimal stopping: A pure exploration analysis

Optimal Change-point Testing for High-dimensional Linear Models with Temporal Dependence