AUTHOR=Wells Lindsay , Bednarz Tomasz TITLE=Explainable AI and Reinforcement Learning—A Systematic Review of Current Approaches and Trends JOURNAL=Frontiers in Artificial Intelligence VOLUME=Volume 4 - 2021 YEAR=2021 URL=https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2021.550030 DOI=10.3389/frai.2021.550030 ISSN=2624-8212 ABSTRACT=Research into Explainable Artificial Intelligence (XAI) has been increasing in recent years as a response to the need for increased transparency and trust in AI, given its use in sensitive domains with societal, ethical, and safety implications. Work in XAI has primarily focused on Machine Learning (ML) for classification, decision, or action, with detailed systematic reviews already undertaken. This review looks to explore current directions and approaches for XAI in a subset of ML, Deep Reinforcement Learning (DRL). From 520 search results, 25 studies (including 5 snowball sampled) are reviewed, highlighting visualization, query-based explanations, policy summarization, human-in-the-loop collaboration, and verification as trends in this area. Limitations in the studies are presented, particularly a lack of user studies, and the prevalence of toy-examples or implementations that are unlikely to scale. Areas for future study are identified, including immersive visualization, and symbolic representation.