Real-time Distributed Optimization of Traffic Signal Timing

Abstract

Leveraging recent advancements in distributed optimization and reinforcement learning, and the growing connectivity and computational capability of vehicles and infrastructure, we propose to advance real-time adaptive signal control via distributed control and optimization. This report consists of three parts. Part 1 develops distributed algorithms for solving a traffic signal timing optimization problem, which is formulated as a mixed-integer programming model. Specifically, the alternating direction method of multipliers (ADMM) is employed, and a two-stage stochastic cell transmission model (CTM) that considers the uncertainty of traffic demand and vehicle turning ratios is considered. Part 2 proposes a framework that utilizes reinforcement learning to optimize a max pressure controller considering the phase switching loss. The max pressure control is modified by introducing a switching curve, and the proposed control method is proved throughput-optimal in a store-and-forward network. Then the theoretical control policy is extended by using a distributed approximation and position-weighted pressure so that the policy-gradient reinforcement learning algorithms can be utilized to optimize the parameters in the policy network including the switching curve and the weigh curve. Part 3 applies reinforcement learning to traffic signal control in a multi-agent scheme, considering the data availability and implementability. The information extracted from traffic cameras is used to define the state of the agents; the action design is aligned with the NEMA dual-ring convention and bounded by a safety constraint, and the coordination is achieved by a shared reward structure among agents.