Machine Learning for Safety-Critical Applications: Opportunities, Challenges, and a Research Agenda

Advances in artificial intelligence, and specifically in machine learning, are enabling new capabilities across nearly every sector of the economy. Many of these applications - such as automated vehicles, the power grid, or surgical robots - are safety critical: where malfunctions can result in harm to people, the environment, or property. While machine learning is already being deployed to enhance the capabilities of some physical systems, extending the rigorous practices of safety engineering to include machine learning components brings significant challenges.

Machine Learning for Safety-Critical Applications explores ways to safely integrate machine learning into physical systems and presents research priorities for improving safety, testing, and evaluation. This report finds that designing machine learning algorithms in a way that aligns with safety engineering standards will require changes in research, training, and engineering practice - as well as a shift away from focusing on algorithmic performance in isolation.