Theory, Extensions, and Applications of Recursive Least Squares

Abstract

Recursive least squares (RLS) is a foundational algorithm in systems and control theory for the online identification of fixed parameters. However, a critical flaw of RLS is its inability to track time-varying parameters. RLS with a forgetting factor can track time-varying parameters, but is fragile without persistently exciting (PE) data. This dissertation develops theory and new extensions of RLS, to accomplish with what these classical algorithms cannot. These results are applied to online system identification for adaptive control.

Following an introductory chapter, the second chapter presents three derivations of RLS. These derivations introduce important concepts used throughout this dissertation. The third chapter introduces RLS with a forgetting factor, or exponential forgetting (EF) RLS, and presents guaranteed covariance bound for EF-RLS with persistent excitation. These guaranteed bounds serve as a baseline for new algorithms presented later in this dissertation. The fourth chapter addresses efficient RLS identification of parameters in a matrix structure and reveals a tradeoff between computational complexity and algorithm generality. This result is used to dramatically speed up online identification of multi-input-multi-output (MIMO) input/output models in an adaptive control scheme. The fifth chapter studies identification of MIMO input/output models using RLS when model order is higher than system order. We show that in this scenario, data cannot be persistent excitation and address convergence without PE.

The sixth and seventh chapters introduce novel extensions of RLS for situations without PE. The sixth chapter presents Subspace of Information Forgetting (SIFt) RLS, a new extension of RLS designed for scenarios when particular directions are excited and others are not. This is accomplished by forgetting in only in the subspace of information, allowing for bounded covariance without PE. The seventh chapter introduces exponential resetting and cyclic resetting RLS, two algorithms which address periods of high excitation and periods of low excitation. These algorithms prevent covariance windup experienced by EF-RLS during periods of low excitation.

Finally, the eighth and ninth chapters concern the unification of RLS extensions from the literature. The eighth chapter provides an overarching framework for RLS extensions and provides sufficient conditions for stability and robustness of parameter estimation error. The ninth chapter builds on the eighth chapter to show that extensions of RLS are also special cases of the Kalman filter. This motivates a new class of adaptive Kalman filters for state estimation in the presence of unmodeled disturbances.