Learning, Control, and Reduction for Markov Jump Systems

Abstract

Many real-world phenomena or systems with temporally changing dynamics can be effectively characterized by time-varying models — typically ensembles of simple models (modes) among which the active mode switches over time. Coupling different modes through switching increases the model capacity so that more complex behaviors can be explained, but this meanwhile brings new challenges: one cannot simply study the entire model by looking at individual modes separately, and the coupling structures need to be factored in. Because of this, recent advances for simple time-invariant models such as data-driven methodologies and sharp finite-sample guarantees are yet to be generalized to time-varying models.

An important class of time-varying models is given by the Markov jump linear systems (MJSs), where each MJS is made up of a collection of linear modes and a Markov chain modeling their switching. MJSs strike a good balance between describing complex temporal variations in dynamics and possessing simple solutions to many classical control problems. This dissertation focuses on MJSs and revisits classical problems including the identification, data-driven control, and model reduction for MJSs. With recent advances in machine learning, optimization, and statistics, this work seeks to address challenges incurred by the mode switching in MJSs and bring new perspectives to these problems. For an MJS with unknown dynamics, we propose an identification scheme to develop its model, which requires only a single data trajectory and is guaranteed to have near-optimal sample complexity. Then, by establishing novel perturbation analysis, two classical control problems, certainty equivalent control and adaptive control, are studied with focus on solving for the linear quadratic regulator (LQR) problems. For the latter, the proposed adaptive control method invokes our identification scheme and is guaranteed to achieve performance with sublinear regret. Sometimes, a designed or learned MJS may suffer from complexities incurred by the sheer number of modes. Using clustering techniques from unsupervised learning, we develop a model reduction scheme that constructs a reduced-mode MJS by grouping modes with similar dynamics, which provably and empirically approximates the original system.