Nonlinear Markov renewal theory with applications to sequential analysis.

Abstract

The study of boundary crossings of stochastic processes has proven extremely useful in probability and statistics. Much attention has been paid to providing conditions under which the overshoot of a process converges in distribution as the boundary becomes large. Such results, called renewal theorems below, are the focus of this thesis. In particular, the thesis focuses on nonlinear Markov renewal theorems, which are renewal theorems for processes that are asymptotically close, in an appropriate sense, to Markov random walks. (A Markov random walk is a partial sum process in which the summands are conditionally independent given a Markov chain.) Such theorems are necessary for the statistical analysis of many sequential procedures involving dependent data. The first type of asymptotic closeness considered requires that the process of interest (at a given time) be expressible as the sum of a Markov random walk and a slowly changing perturbation term. In this setting a nonlinear renewal theorem is proved, i.e., it is shown that the overshoot of the process of interest has the same limiting distribution as does the overshoot of the Markov random walk. The theorem is applied to repeated significance testing for an autoregressive process and to sequential testing in clinical trials with adaptive allocation. The second type of asymptotic closeness is more subtle. It requires that the finite-dimensional conditional distributions of the process of interest converge (in the Prokhorov metric) to those of an appropriate Markov random walk. Proving renewal theorems in this case requires a much more delicate argument. Such theorems are applicable to sequential testing in clinical trials where allocation is adaptive and incorporates ethical costs, and to the analysis of machine breakdowns where the machine is adjusted at the time of each breakdown. In both cases, conditions are given under which the joint distribution of the overshoot and the Markov chain at the time of the boundary crossing converges. Also, in the process of proving the results, the convergence in Kesten's Markov renewal theorem is shown to hold uniformly on compact sets. This result may prove to be of independent interest.