Safe Exploration

Summary

Safe exploration in reinforcement learning aims to enable agents to learn and optimize their behavior while avoiding dangerous or undesirable states. This involves developing algorithms and techniques that allow for effective exploration of the environment while maintaining safety constraints. Key approaches include using uncertainty estimates to identify potentially unsafe areas, incorporating formal safety constraints into the learning process, leveraging model-based planning to anticipate and avoid unsafe states, and employing separate task and recovery policies. Some methods use offline data or simulations to learn about unsafe regions before deploying agents in the real world. Others utilize probabilistic shields or reachability analysis to provide safety guarantees. The challenge lies in balancing the need for exploration to improve performance with the requirement to avoid catastrophic failures or constraint violations. Safe exploration is particularly crucial for deploying reinforcement learning in real-world, safety-critical applications.

Research Papers

• Verifiably Safe Exploration for End-to-End Reinforcement Learning
• AI Safety Gridworlds
• Safe Reinforcement Learning with Model Uncertainty Estimates
• Safe Reinforcement Learning by Imagining the Near Future
• Active Reinforcement Learning with Monte-Carlo Tree Search
• Curiosity Killed or Incapacitated the Cat and the Asymptotically Optimal Agent
• Lyapunov-based Safe Policy Optimization for Continuous Control
• Leave no Trace Learning to Reset for Safe and Autonomous Reinforcement Learning
• Recovery RL Safe Reinforcement Learning with Learned Recovery Zones
• Safe Reinforcement Learning via Probabilistic Shields
• Safely Probabilistically Complete Real-Time Planning and Exploration in Unknown Environments