Safe Reinforcement Learning

A compilation of recent machine learning papers focused on safe reinforcement learning, currently spanning from 2017 to 2022. If you would like to contribute additional papers or update the list, please feel free to do so on the our Safe-RL GitHub page.

  1. Safe Reinforcement Learning via Curriculum Induction, Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, and Alekh Agarwal, NeurIPS 2020
  2. Safe Reinforcement Learning with Natural Language Constraints, Tsung-Yen Yang, Michael Hu, Yinlam Chow, Peter J. Ramadge, and Karthik Narasimhan, NeurIPS 2021
  3. Adversarial Policies: Attacking Deep Reinforcement Learning, Adam Gleave, Michael Dennis, Cody Wild, Neel Kant, Sergey Levine, and Stuart Russell, ICLR 2020
  4. Reward constrained policy optimization, Chen Tessler, Daniel J. Mankowitz, and Shie Mannor, ICLR 2019
  5. Safe Reinforcement Learning by Imagining the Near Future, Garrett Thomas, Yuping Luo, and Tengyu Ma, NeurIPS 2021
  6. Constrained Policy Optimization via Bayesian World Models, Yarden As, Ilnura Usmanova, Sebastian Curi and Andreas Krause, ICLR 2022
  7. Constrained Policy Optimization, Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel, ICML 2017
  8. Responsive Safety in Reinforcement Learning by PID Lagrangian Methods Adam, Adam Stooke, Joshua Achiam, and Pieter Abbeel, ICML 2020
  9. There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning, Nathan Grinsztajn, Johan Ferret, Olivier Pietquin, Philippe Preux, and Matthieu Geist, NeurIPS 2021
  10. Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble, Gaon An, Seungyong Moon, Jang-Hyun Kim, and Hyun Oh Song, NeurIPS 2021
  11. Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations, Yuping Luo, and Tengyu Ma, NeurIPS 2021
  12. Teachable Reinforcement Learning via Advice Distillation, Olivia Watkins, Trevor Darrell, Pieter Abbeel, Jacob Andreas, and Abhishek Gupta, NeurIPS 2021
  13. Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings, Jesse Zhang, Brian Cheung, Chelsea Finn, Sergey Levine, and Dinesh Jayaraman, ICML 2020
  14. Verifiable Reinforcement Learning via Policy Extraction, Osbert Bastani, Yewen Pu, and Armando Solar-Lezama, NeurIPS 2018
  15. Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods, Seohong Park, Jaekyeom Kim, and Gunhee Kim, NeurIPS 2021
  16. Risk-averse Offline Reinforcement Learning, Núria Armengol Urpí, Sebastian Curi, and Andreas Krause, ICLR 2021
  17. Quantifying Generalization in Reinforcement Learning, Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman, ICML 2019
  18. Deep Reinforcement Learning from Human Preferences, Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, and Dario Amodei, NeurIPS 2017
  19. A Lyapunov-based Approach to Safe Reinforcement Learning, Yinlam Chow, Ofir Nachum, Edgar Duenez-Guzman, and Mohammad Ghavamzadeh, NeurIPS 2018
  20. Lyapunov-based Safe Policy Optimization for Continuous Control, Yinlam Chow, Ofir Nachum, Aleksandra Faust, Edgar Duenez-Guzman, and Mohammad Ghavamzadeh, RL4RealLife Workshop in ICML 2019
  21. Constrained Variational Policy Optimization for Safe Reinforcement Learning, Zuxin Liu, Zhepeng Cen, Vladislav Isenbaev, Wei Liu, Zhiwei Steven Wu, Bo Li, and Ding Zhao, ICML 2022
  22. Safe Reinforcement Learning with Linear Function Approximation, Sanae Amani, Christos Thrampoulidis, and Lin F. Yang, ICML 2021
  23. Reachability Constrained Reinforcement Learning, Dongjie Yu, Haitong Ma, Shengbo Eben Li, and Jianyu Chen, ICML 2022
  24. Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings, Jesse Zhang, Brian Cheung, Chelsea Finn, Sergey Levine, and Dinesh Jayaraman, ICML 2020
  25. First Order Constrained Optimization in Policy Space, Yiming Zhang, Quan Vuong, and Keith W. Ross, NeurIPS 2020
  26. Safe Policy Optimization with Local Generalized Linear Function Approximations, Akifumi Wachi,Yunyue Wei, and Yanan Sui, NeurIPS 2021
  27. Safe Model-based Reinforcement Learning with Stability Guarantees, Felix Berkenkamp, Matteo Turchetta, Angela P. Schoellig, and Andreas Krause, NeurIPS 2017
  28. Neurosymbolic Reinforcement Learning with Formally Verified Exploration, Greg Anderson, Abhinav Verma, Isil Dillig, and Swarat Chaudhuri, NeurIPS 2020
  29. Conservative Offline Distributional Reinforcement Learning, Yecheng Jason Ma, Dinesh Jayaraman, and Osbert Bastani, NeurIPS 2021
  30. Continuous Doubly Constrained Batch Reinforcement Learning, Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Pratik Chaudhari, and Alexander J. Smola, NeurIPS 2021
  31. Risk-Aware Transfer in Reinforcement Learning using Successor Features, Michael Gimelfarb, André Barreto, Scott Sanner, and Chi-Guhn Lee, NeurIPS 2021
  32. Teachable Reinforcement Learning via Advice Distillation, Olivia Watkins, Trevor Darrell, Pieter Abbeel, Jacob Andreas, and Abhishek Gupta, NeurIPS 2021
  33. There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning, Nathan Grinsztajn, Johan Ferret, Olivier Pietquin, Philippe Preux, and Matthieu Geist, NeurIPS 2021
  34. Provably Good Batch Reinforcement Learning Without Great Exploration, Yao Liu, Adith Swaminathan, Alekh Agarwal, and Emma Brunskill, NeurIPS 2020
  35. Convergent Policy Optimization for Safe Reinforcement Learning, Ming Yu, Zhuoran Yang, Mladen Kolar, and Zhaoran Wang, NeurIPS 2019
  36. Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints, Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Devidze, and Adish Singla, NeurIPS 2019
  37. Security Analysis of Safe and Seldonian Reinforcement Learning Algorithm, A. Pinar Ozisik, and Philip S. Thomas, NeurIPS 2020
  38. Leave No Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning, Benjamin Eysenbach, Shixiang Gu, Julian Ibarz, and Sergey Levine, ICLR 2018
  39. Conservative Safety Critics for Exploration, Homanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkurti, and Animesh Garg, ICLR 2021
  40. Safe Reinforcement Learning Using Advantage-Based Intervention Nolan, Nolan Wagener, Byron Boots, and Ching-An Cheng ICML 2021
  41. Accelerating Safe Reinforcement Learning with Constraint-mismatched Baseline Policies, Tsung-Yen Yang, Justinian Rosca, Karthik Narasimhan,and Peter J. Ramadge,
  42. Constrained Reinforcement Learning Has Zero Duality Gap, Santiago Paternain, Luiz F. O. Chamon, Miguel Calvo-Fullana and Alejandro Ribeiro, NeurIPS 2019
  43. Balancing Constraints and Rewards with Meta-gradient D4PG, Dan A. Calian, Daniel J. Mankowitz, Tom Zahavy, Zhongwen Xu, Junhyuk Oh, Nir Levine, and Timothy Mann, ICLR 2021
  44. Conservative Q-Learning for Offline Reinforcement Learning, Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine, NeurIPS 2020)
  45. Sauté RL: Almost Surely Safe Reinforcement Learning Using State Augmentation, Aivar Sootla, Alexander I. Cowen-Rivers, Taher Jafferjee, Ziyan Wang, David Mguni, Jun Wang, and Haitham Bou-Ammar, ICML 2022
  46. Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives, Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng, Sergey Levine, and Yoshua Bengio, ICLR 2020
  47. Density Constrained Reinforcement Learning, Zengyi Qin, Yuxiao Chen, and Chuchu Fan, ICML 2021