Multi-Agent Reinforcement Learning Autonomous Driving Highway On-Ramp Merge

Abstract

Autonomous driving is expected to become more common in the future. Autonomous vehicles operate today in limited use cases like highway driving and in major cities as robotaxis, but full L5 operation is yet to be achieved. Functions like fully autonomous highway ramp entry must be available, safe, and reliably robust in a provable way to bridge the gap to enable full autonomy. Towards this goal, my research produces three main contributions: a fundamental study of on-ramp merging defining specific ever-present behaviors, a standard test framework for evaluating merging performance, and a multi-agent DRL simulation that has learned to operate with nearly ideal collision avoidance performance. The fundamental study shows behaviors and limitations that exist with all merging. A standard test framework I developed evaluates performance and compares different approaches. The virtual environment of the multiagent DRL uses self-play with simulated data where merging vehicles safely learn to control longitudinal position during a taper-type merge. The initial simulation setup is a two-vehicle merge-traffic pair, then it is progressively scaled up to a full merge scene. The simulation results show nearly perfect performance that is likely best-in-class if it were able to be compared against other research using the standard evaluation framework.