Drift Counteraction Optimal Control: Theory and Applications to Autonomous Cars and Spacecraft

Abstract

Many engineering systems are subject to persistent disturbances or dynamics that cause the process variables to drift. This dissertation studies the problem of how to control such systems in order to maximize the time or total yield before prescribed constraints are violated. This problem is referred to as drift counteraction optimal control (DCOC) since the controller may be viewed as counteracting drift in order to delay constraint violation.

The first part of this dissertation focuses on deterministic DCOC problems. Conditions for the existence of a solution are derived and an optimal control strategy is characterized in terms of the value function. New algorithms based on dynamic programming (DP), approximate dynamic programming (ADP), and model predictive control (MPC) are developed to obtain solutions or good-quality approximations of solutions. In terms of DP, an enhanced version of the value iteration (VI) algorithm is proposed that converges to the value function faster than conventional VI in a numerical setting. Based on the enhanced VI algorithm, an ADP approach is obtained to mitigate the curse of dimensionality. Another DP-based algorithm, referred to as base-trajectory VI, is proposed, which converges to the value function by gradually connecting pieces of an optimal control policy. A mixed-integer nonlinear program is derived that obtains open-loop solutions to deterministic DCOC problems and good-quality suboptimal solutions are obtained with a similar nonlinear program without integer variables. In the linear systems case, a mixed-integer linear program (MILP) and a standard linear program (LP) are formulated to obtain open-loop solutions and good-quality approximations of solutions, respectively. Based on linear model approximation, the MILP and LP are used to implement an MPC strategy that can be effective in DCOC of nonlinear systems. New applications of deterministic DCOC are proposed with a focus on spacecraft control and several numerical case studies are treated.

In the second part of this dissertation, the assumption of perfect information is relaxed and the developments for the case of deterministic DCOC are extended to the case of stochastic systems. Conditions are derived under which the average first exit-time and value function are bounded from above, which are necessary conditions for the existence of a solution to stochastic DCOC problems. Further conditions are provided that guarantee the existence of a solution and an optimal control policy is characterized. The enhanced VI algorithm for the deterministic case is extended to stochastic systems, where the proof of convergence requires a different approach. Furthermore, an ADP approach is presented that mitigates the curse of dimensionality. Another contribution is a novel tree-based stochastic MPC approach to solve stochastic DCOC problems. A scenario tree with a specified number of tree nodes is used to encode the most likely system behavior, where each path on the tree corresponds to a distinct disturbance scenario. For linear discrete-time systems with an additive random disturbance, an MILP obtains solutions arbitrarily close to the optimal solution for a sufficient number of tree nodes. In order to compensate for an incomplete scenario tree and/or unmodeled effects, feedback is provided by recomputing the MILP solution over a receding time horizon based on the current disturbance and state variables. New applications of stochastic DCOC are introduced with a focus on automated and autonomous driving and a variety of numerical case studies of such DCOC problems are treated in this dissertation.