On Predictive Linear Gaussian Models.

Abstract

Models are used by artificial agents to make predictions about the future; agents then use these predictions to modify their behavior. In many cases, these models are not known a priori and so the agent must learn a model through experience with a system. At the core of most models is the concept of state--an estimate of the current situation of the world from which a model's predictions are derived.

A recent development in the study of models is the predictive state} model. Predictive state models use predictions about potential future events as their state, as opposed to unobserved, unobservable variables, as in most traditional models. For example, a traditional model may represent a robot's location using latitude and longitude, which is unobservable without a GPS unit. A predictive state model of the same robot might represent its location with two events like "If I traveled forward 4 feet I would hit a wall" and "If I turned right and traveled forward 6 feet I would move into a hallway."

This dissertation presents two models that expand the limits of predictive state models, which had mostly modeled dynamical systems with discrete, scalar-valued observations, with linear predictions of future events. The first model, the e-test predictive state representation (EPSR), is the first nonlinear predictive state model that can be used to model a large class of dynamical systems. The EPSR models deterministic systems with discrete actions and observations, and is sometimes exponentially smaller than the equivalent model with linear predictions.

The second model is the predictive linear Gaussian model (PLG), which models dynamical systems with continuous vector-valued actions and observations. I present theoretical results that show the PLG is representationally equivalent to the linear dynamical system (LDS), a popular traditional model, and that the parameter estimation algorithm I present is consistent--that is, in the limit of infinite data, it produces a correct model. I also apply this algorithm to a) a number of artificial, randomly generated systems and b) a real-world traffic prediction problem, and show that it performs well compared to expectation maximization, a parameter estimation algorithm for the LDS.