View Item 

Show simple item record

Improving autonomous vehicle in-traffic safety using learning-based action governor
dc.contributor.authorHan, Kyoungseok
dc.contributor.authorLi, Nan
dc.contributor.authorTseng, Eric
dc.contributor.authorFilev, Dimitar
dc.contributor.authorKolmanovsky, Ilya
dc.contributor.authorGirard, Anouck
dc.date.accessioned2022-07-05T21:02:32Z
dc.date.available2023-07-05 17:02:30en
dc.date.available2022-07-05T21:02:32Z
dc.date.issued2022-06
dc.identifier.citationHan, Kyoungseok; Li, Nan; Tseng, Eric; Filev, Dimitar; Kolmanovsky, Ilya; Girard, Anouck (2022). "Improving autonomous vehicle in-traffic safety using learning-based action governor." Advanced Control for Applications: Engineering and Industrial Systems 4(2): n/a-n/a.
dc.identifier.issn2578-0727
dc.identifier.issn2578-0727
dc.identifier.urihttps://hdl.handle.net/2027.42/173001
dc.description.abstractThe Action Governor (AG) is a supervisory scheme augmenting a nominal control system in order to enhance the system’s safety and performance. It acts as an action filter, monitoring the action commands generated by the nominal control policy and adjusting the ones that might lead to undesirable system behavior. In this article, we present an approach based on learning to developing an AG for autonomous vehicle (AV) decision policies to improve their safety for operating in mixed-autonomy traffic (i.e., traffic involving both AVs and human-operated vehicles (HVs)). To achieve this, we demonstrate that it is possible to train the AG in a traffic simulator that is capable of representing in-traffic interactions among AVs and HVs. We illustrate the effectiveness of this learning-based AG approach to improving AV in-traffic safety through simulation case studies.The Figure shows a learning-based approach to designing a safety supervisor for improving AV in-traffic safety based on the Action Governor (AG) framework. The AG is an add-on scheme augmenting a nominal control loop to enhance the system’s safety and performance. The learning approach is based on identification and recording of unviable state and action pairs rather than only the ones that cause immediate safety violations.
dc.publisherMIT Press
dc.publisherWiley Periodicals, Inc.
dc.subject.otheraction governor
dc.subject.otherautonomous vehicle
dc.subject.otherlearning-based control
dc.subject.otherreinforcement learning
dc.titleImproving autonomous vehicle in-traffic safety using learning-based action governor
dc.typeArticle
dc.rights.robotsIndexNoFollow
dc.subject.hlbsecondlevelIndustrial and Operations Engineering
dc.subject.hlbtoplevelEngineering
dc.description.peerreviewedPeer Reviewed
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/173001/1/adc2101.pdf
dc.description.bitstreamurlhttp://deepblue.lib.umich.edu/bitstream/2027.42/173001/2/adc2101_am.pdf
dc.identifier.doi10.1002/adc2.101
dc.identifier.sourceAdvanced Control for Applications: Engineering and Industrial Systems
dc.identifier.citedreferenceAmes AD, Xu X, Grizzle JW, Tabuada P. Control barrier function based quadratic programs for safety critical systems. IEEE Trans Automat Contr. 2016; 62 ( 8 ): 3861 - 3876.
dc.identifier.citedreferenceLi Z, Kalabić U, Chu T. Safe reinforcement learning: learning with supervision using a constraint-admissible set. Proceedings of the 2018 American Control Conference (ACC); 2018:6390-6395.
dc.identifier.citedreferenceCheng R, Orosz G, Murray RM, Burdick JW. End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. Proceedings of the AAAI Conference on Artificial Intelligence; Vol. 33, 2019:3387-3395.
dc.identifier.citedreferenceChen Y, Hereid A, Peng H, Grizzle J. Enhancing the performance of a safe controller via supervised learning for truck lateral control. J Dyn Syst Meas Control. 2019; 141 ( 10 ): 06 2019.
dc.identifier.citedreferenceLi Y, Li N, Tseng HE, Girard A, Filev D, Kolmanovsky I. Safe reinforcement learning using robust action governor. Proceedings of the Learning for Dynamics and Control (L4DC) Conference; 2021:1093-1104.
dc.identifier.citedreferenceHan T, Nageshrao S, Filev DP, Özgüner Ü. An online evolving framework for advancing reinforcement-learning based automated vehicle control. IFAC-PapersOnLine. 2020; 53 ( 2 ): 8118 - 8123.
dc.identifier.citedreferenceLiu K, Li N, Rizzo D, Garone E, Kolmanovsky I, Girard A. Model-free learning to avoid constraint violations: an explicit reference governor approach. Proceedings of the 2019 American Control Conference (ACC); 2019:934-940.
dc.identifier.citedreferenceLiu K, Li N, Kolmanovsky I, Rizzo D, Girard A. Model-free learning for safety-critical control systems: a reference governor approach. Proceedings of the 2020 American Control Conference (ACC); 2020:943-949.
dc.identifier.citedreferenceLi N, Han K, Girard A, Tseng HE, Filev D, Kolmanovsky I. Action governor for discrete-time linear systems with non-convex constraints. IEEE Control Syst Lett. 2020; 5 ( 1 ): 121 - 126.
dc.identifier.citedreferenceLi Y, Li N, Tseng HE, Girard A, Filev D, Kolmanovsky I. Robust action governor for discrete-time piecewise affine systems with additive disturbances. IEEE Control Syst Lett. 2021.
dc.identifier.citedreferenceRomdlony MZ, Jayawardhana B. Stabilization with guaranteed safety using control Lyapunov–Barrier function. Automatica. 2016; 66: 39 - 47.
dc.identifier.citedreferenceAmes AD, Coogan S, Egerstedt M, Notomista G, Sreenath K, Tabuada P. Control barrier functions: theory and applications. Proceedings of the 18th European Control Conference (ECC); 2019:3420-3431.
dc.identifier.citedreferenceLi N, Oyler DW, Zhang M, Yildiz Y, Kolmanovsky I, Girard AR. Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans Control Syst Technol. 2017; 26 ( 5 ): 1782 - 1797.
dc.identifier.citedreferenceGarone E, Di Cairano S, Kolmanovsky I. Reference and command governors for systems with constraints: a survey on theory and applications. Automatica. 2017; 75: 306 - 328.
dc.identifier.citedreferenceGray A, Gao Y, Lin T, Hedrick JK, Tseng HE, Borrelli F. Predictive control for agile semi-autonomous ground vehicles using motion primitives. Proceedings of the 2012 American Control Conference (ACC); 2012:4239-4244.
dc.identifier.citedreferenceSutton RS, Barto AG. Reinforcement Learning: An Introduction. MIT Press; 2018.
dc.identifier.citedreferenceYoo JH, Langari R. Stackelberg game based model of highway driving. Proceedings of the 2012 Dynamic Systems and Control Conference; Vol. 45295, 2012:499-508.
dc.identifier.citedreferenceClaussmann L, Carvalho A, Schildbach G. A path planner for autonomous driving on highways using a human mimicry approach with binary decision diagrams. Proceedings of the 2015 European Control Conference (ECC); 2015:2976-2982.
dc.identifier.citedreferenceLi N, Zhang M, Yildiz Y, Kolmanovsky I, Girard A. Game theory-based traffic modeling for calibration of automated driving algorithms. Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions. Springer; 2019: 89 - 106.
dc.identifier.citedreferenceChen C, Seff A, Kornhauser A, Xiao J. Deepdriving: learning affordance for direct perception in autonomous driving. Proceedings of the IEEE International Conference on Computer Vision; 2015:2722-2730.
dc.identifier.citedreferenceStahl DO, Wilson PW. On players’ models of other players: theory and experimental evidence. Games Econ Behav. 1995; 10 ( 1 ): 218 - 254.
dc.identifier.citedreferenceSu G, Li N, Yildiz Y, Girard A, Kolmanovsky I. A traffic simulation model with interactive drivers and high-fidelity car dynamics. IFAC-PapersOnLine. 2019; 51 ( 34 ): 384 - 389.
dc.identifier.citedreferenceAlbaba M, Yildiz Y, Li N, Kolmanovsky I, Girard A. Stochastic driver modeling and validation with traffic data. Proceedings of the 2019 American Control Conference (ACC); 2019:4198-4203.
dc.identifier.citedreferenceJaakkola T, Singh SP, Jordan MI. Reinforcement learning algorithm for partially observable Markov decision problems. Adv Neural Inf Process Syst. 1995; 345 - 352.
dc.identifier.citedreferenceFraade-Blanar L, Blumenthal MS, Anderson JM, Kalra N. Measuring automated vehicle safety: forging a framework; 2018.
dc.identifier.citedreferenceLiu K, Li N, Kolmanovsky I, Rizzo D, Girard A. Tanker truck rollover avoidance using learning reference governor. SAE Int J Adv Current Pract Mob. 2021; 3: 1385 - 1394.
dc.identifier.citedreferenceYeong DJ, Velasco-Hernandez G, Barry J, Walsh J. Sensor and sensor fusion technology in autonomous vehicles: a review. Sensors. 2021; 21 ( 6 ): 2140.
dc.identifier.citedreferenceVan Brummelen J, O’Brien M, Gruyer D, Najjaran H. Autonomous vehicle perception: the technology of today and tomorrow. Transp Res C Emerg Technol. 2018; 89: 384 - 406.
dc.identifier.citedreferenceLiu L, Lu S, Zhong R, et al. Computing systems for autonomous driving: state of the art and challenges. IEEE Internet Things J. 2020; 8 ( 8 ): 6469 - 6486.
dc.identifier.citedreferenceCampbell M, Egerstedt M, How JP, Murray RM. Autonomous driving in urban environments: approaches, lessons and challenges. Philos Trans R Soc A Math Phys Eng Sci. 1928; 2010 ( 368 ): 4649 - 4672.
dc.identifier.citedreferenceErsal T, Kolmanovsky I, Masoud N, et al. Connected and automated road vehicles: state of the art and future challenges. Veh Syst Dyn. 2020; 58 ( 5 ): 672 - 704.
dc.identifier.citedreferenceBadue C, Guidolini R, Carneiro RV, et al. Self-driving cars: a survey. Expert Syst Appl. 2021; 165: 113816.
dc.identifier.citedreferenceSchwarting W, Alonso-Mora J, Rus D. Planning and decision-making for autonomous vehicles. Annu Rev Control Robot Auton Syst. 2018; 1: 187 - 210.
dc.identifier.citedreferenceClaussmann L, Revilloud M, Gruyer D, Glaser S. A review of motion planning for highway autonomous driving. IEEE Trans Intell Transp Syst. 2019; 21 ( 5 ): 1826 - 1848.
dc.identifier.citedreferenceKhayatian M, Mehrabian M, Andert E, et al. A survey on intersection management of connected autonomous vehicles. ACM Trans Cyber-Phys Syst. 2020; 4 ( 4 ): 1 - 27.
dc.identifier.citedreferenceSadigh D, Sastry S, Seshia SA, Dragan AD. Planning for autonomous cars that leverage effects on human actions. Robot Sci Syst. 2016; 2: 1 - 9.
dc.identifier.citedreferenceSankar GS, Han K. Adaptive robust game-theoretic decision making strategy for autonomous vehicles in highway. IEEE Trans Veh Technol. 2020.
dc.identifier.citedreferenceZhang M, Li N, Girard A, Kolmanovsky I. A finite state machine based automated driving controller and its stochastic optimization. Proceedings of the 2017 Dynamic Systems and Control Conference; Vol. 58288, 2017:V002T07A002.
dc.identifier.citedreferenceCensi A, Slutsky K, Wongpiromsarn T, et al Liability, ethics, and culture-aware behavior specification using rulebooks. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA); 2019:8536-8542.
dc.identifier.citedreferenceDing J, Li L, Peng H, Zhang Y. A rule-based cooperative merging strategy for connected and automated vehicles. IEEE Trans Intell Transp Syst. 2019; 21 ( 8 ): 3436 - 3446.
dc.identifier.citedreferenceXiao W, Mehdipour N, Collin A,et al. Rule-based optimal control for autonomous driving. Proceedings of the 12th ACM/IEEE International Conference on Cyber-Physical Systems; 2021:143-154.
dc.identifier.citedreferenceZhang J, Cho K. Query-efficient imitation learning for end-to-end simulated driving. Proceedings of the 31st AAAI Conference on Artificial Intelligence; 2017:2891-2897.
dc.identifier.citedreferenceCodevilla F, Müller M, López A, Koltun V, Dosovitskiy A. End-to-end driving via conditional imitation learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA); 2018:4693-4700.
dc.identifier.citedreferenceChen J, Yuan B, Tomizuka M. Deep imitation learning for autonomous driving in generic urban scenarios with enhanced safety. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2019:2884-2890.
dc.identifier.citedreferenceSallab AE, Abdou M, Perot E, Yogamani S. Deep reinforcement learning framework for autonomous driving. Electron Imaging. 2017; 2017 ( 19 ): 70 - 76.
dc.identifier.citedreferenceYou C, Lu J, Filev D, Tsiotras P. Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning. Robot Auton Syst. 2019; 114: 1 - 18.
dc.identifier.citedreferenceNageshrao S, Tseng HE, Filev D. Autonomous highway driving using deep reinforcement learning. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC); 2019:2326-2331.
dc.identifier.citedreferenceLi H, Li N, Kolmanovsky I, Girard A. Energy-efficient autonomous vehicle control using reinforcement learning and interactive traffic simulations. Proceedings of the 2020 American Control Conference (ACC); 2020:3029-3034.
dc.identifier.citedreferenceDuan J, Li SE, Guan Y, Sun Q, Cheng B. Hierarchical reinforcement learning for self-driving decision-making without reliance on labelled driving data. IET Intell Transp Syst. 2020; 14 ( 5 ): 297 - 305.
dc.identifier.citedreferenceChen J, Li SE, Tomizuka M. Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning. IEEE Trans Intell Transp Syst. 2021.
dc.identifier.citedreferenceChen D, Jiang L, Wang Y, Li Z.. Autonomous driving using safe reinforcement learning by incorporating a regret-based human lane-changing decision model. Proceedings of the 2020 American Control Conference (ACC); 2020:4355-4361.
dc.identifier.citedreferenceBaheri A, Nageshrao S, Tseng HE, Kolmanovsky I, Girard A, Filev D. Deep reinforcement learning with enhanced safety for autonomous highway driving. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV); 2020:1550-1555.
dc.working.doiNOen
dc.owningcollnameInterdisciplinary and Peer-Reviewed


Files in this item

Name:
adc2101.pdf
Size:
2.470MB
Format:
PDF
View/Open
Name:
adc2101_am.pdf
Size:
3.414MB
Format:
PDF
View/Open

Show simple item record

Remediation of Harmful Language

The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.

Accessibility

If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.