Autonomous Orbital Slot Maintenance with Impulsive Maneuvers and Reinforcement Learning

Abstract

This paper investigates impulsive maneuver-based control methods for satellite orbital slot maintenance in the presence of dynamic variations. The objective is to develop an efficient controller that ensures the satellite remains within a designated region around a reference trajectory, the slot, while minimizing propellant usage, facilitating autonomous onboard control. The controller is designed using Clohessy-Wiltshire terminal guidance for a linearized satellite dynamics model relative to the slot. The Q-learning algorithm is utilized in the decision-making process for selecting the optimal target point, after which the appropriate control input is determined through Clohessy-Wiltshire terminal guidance. The inclusion of atmospheric drag and gravity model differences adds complexity to determining the optimal target point for the maneuvers. The learning process produces a controller that favors the target point closest to the position where the satellite leaves the slot, reducing propellant costs by approximately 15.9% compared to when Q-learning was not used. These results underscore the effectiveness of Q-learning for maneuver optimization, consistently achieving greater rewards compared to returning to the slot center, implying reduced propellant consumption and enhanced position maintenance. Furthermore, the computational time remains within practical limits, making this approach a viable option for autonomous onboard control.