Machine Learning (ML) has the potential to enable a new generation of safety-related systems in a wide range of domains including aviation, healthcare, mining, rail and road transport. However, ML and its uses pose significant challenges to traditional approaches to assuring safety of systems. This is true both for ML-based decision-support systems and where the ML components are embedded into a larger system, e.g. a semi-autonomous road vehicle. Fundamentally, such ML-based systems transfer decision-making, or the production of recommendations, from humans to computers – and the ML components do not have the contextual and semantic knowledge of the environment that humans do, and on which systems currently rely to ensure safety. At the technical level, the challenges arise from the complexity of the ML algorithms, and from the nature of the development process, including issues of how representative the training data is of the environment of use of the system, and the opacity of many algorithms, especially deep-learning.

This Research Topic aims to address these issues. Contributions are invited that set out systematic approaches to characterizing the problems of ensuring safety of ML-based systems, focusing on technologies, applications domains, or their intersection. Similarly, contributions are invited that set out the state-of-the-art on specific sub-problems or present systematic reviews that help identify future strategies and prospects, for working on these sub-problems. Papers should have a technical core, but multi-disciplinary contributions, e.g. that address social acceptability and usability, are also invited.

Issues and key sub-problems in the scope of the Research Topic include, but are not limited to:

• Identifying the challenges of developing safe ML-based systems, e.g.
- Oversight of safety of ML-based systems
- Understanding safety impact of distributional shift

• Applying systematic safety engineering methods to the system using ML, e.g.
- Identifying ML-related hazards
- Deriving safety requirements on ML components of systems

• Improving/altering the design and development of the ML to enhance safety, e.g. through showing
- Robustness
- Explainability

• ML verification and validation (V&V) to generate evidence to assure safety of ML, e.g.
- Adaptation of traditional software engineering methods
- Newly developed ML-specific methods
- Frameworks for assuring safety of ML-based systems

• Continued safety for ML, e.g.
- Safe, agile updates during operation
- Learning from operational data (rather than pre-deployment data)
- Developing and updating a safety assurance case

• Empirical studies evaluating
- Safety, robustness, explainability, etc.
- Human-machine interaction
- Usability of ML-based systems

• Machine learning in safety engineering and analysis
- Use of ML to generate safety artefacts, e.g., hazard analyses, safety cases
- Use of ML to assist in reviewing safety artefacts

• Ethically-informed development of ML-based systems, including balancing benefits and harms

Finally, contributions may be technically-focused, or address domain-specific problems.

Submit your idea

Impact

Frontiers in Computer Science

Submit your idea

Impact