Computational Inference Algorithms for Spatiotemporal Processes and Other Complex Models

Abstract

Data analysis can be carried out based on a stochastic model that reflects the analyst's understanding of how the system in question behaves. The stochastic model describes where in the system randomness is present and how the randomness plays a role in generating data. The likelihood of the data defined by the model summarizes the evidence provided by observations of the system. Drawing inference from the likelihood of the data, however, can be far from being simple or straightforward, especially in modern statistical data analyses. Complex probability models and big data call for new computational methods to translate the likelihood of data into inference results. In this thesis, I present two innovations in computational inference for complex stochastic models.

The first innovation lies in the development of a method that enables inference on coupled dynamic systems that are partially observed. The high dimensionality of the model that defines the joint distribution of the coupled dynamic processes makes computational inference a challenge. I focus on the case where the probability model is not analytically tractable, which makes the computational inference even more challenging. A mechanistic model of a dynamic process that is defined via a simulation algorithm can lead to analytically intractable models. I show that algorithms that utilize the Markov structure and the mixing property of stochastic dynamic systems can enable fully likelihood based inference for these high dimensional analytically intractable models. I demonstrate theoretically that these algorithms can substantially reduce the computational cost for inference, and the reduction may be orders of magnitude in practice. Spatiotemporal dynamics of measles transmission are inferred from data collected at linked geographic locations, as an illustration that this algorithm can offer an advance in scientific inference.

The second innovation involves a generalization of the framework in which samples from a probability distribution with unnormalized density are drawn using Markov chain Monte Carlo algorithms. The new framework generalizes the widely used Metropolis-Hastings acceptance or rejection strategy. The resulting method is straightforward to implement in a broad range of MCMC algorithms, including the most frequently used ones such as random walk Metropolis, Metropolis adjusted Langevin, Hamiltonian Monte Carlo, or the bouncy particle sampler. Numerical studies show that this new framework enables flexible tuning of parameters and facilitates faster mixing of the Markov chain, especially when the target probability density has complex structure.