Replicable Reinforcement Learning

Speaker 

Jessica Sorrell March 21, 2025. 

Abstract

The replicability crisis in the social, behavioral, and data sciences has led to the formulation of algorithmic frameworks for replicability – i.e., a requirement that an algorithm produce identical outputs (with high probability) when run on two different samples from the same underlying distribution. In control problems such as reinforcement learning, where an agent’s actions can change the underlying distribution, such stability guarantees can be challenging to obtain. We will introduce provably replicable algorithms for reinforcement learning in tabular MDPs, which guarantee that even in stochastic environments, two independent runs of a learning algorithm will converge on precisely the same policy with high probability.