Identification and Quantification of Hazards for autonomous Systems
The increasing automation of technical systems promises many benefits. In the automotive sector, for example, automated driving is seen as a key element in achieving Vision Zero by avoiding human causes of accidents such as drunk driving or driver distraction. However, this is offset by new failure modes and hazards resulting from the introduction of automation in systems where human operators were previously responsible for controlling and monitoring correct and safe operation.
Established safety processes are primarily concerned with assessing functional safety, leaving human operator to select safe maneuvers. With automation in the "driver's seat", it becomes necessary to perform a robust Safety Of The Intended Functionality (SOTIF) assessment to ensure safe planning and execution of maneuvers in all possible environments. In particular, the following classes of problems must be considered:
- Many of these systems operate in an open context that is difficult to fully characterize and is susceptible to disruptive changes in the life cycle of the system.
- Technical perception systems operate differently than human ones, so new hazards can arise in situations that were never considered particularly challenging.
- Decision making procedures based on AI algorithms does not yet exhibit the reliability required for safety-critical applications, while unsafe behavior may be masked due to the inherent problem of explainability.
- The absence of human operators in the loop results in the loss of redundant monitoring capabilities
- There is insufficient field data available to support empirical safety arguments.
To address these issues, we offer two approaches to 1) find common causes of criticality for specific classes of applications and 2) identify and quantitatively assess emerging threat sources for a given system automation
Critical Analysis
Criticality analysis aims at systematically structuring the complexity of the operational domain of automated systems. This includes not only the identification of factors, parameters and scenari-os that have a significant influence on criticality, but also the abstraction and mapping to a manageable set of artifacts that capture and explain the emergence of critical situations for such systems. System engineers can use the criticality analysis before the actual system design, since it precedes the classical design phase. This independence from the actual system design enables reuse of the artifacts, for example, by different stakeholders. During the dev-ops process, analysis can be used to evaluate changes in the operational domain. Criticality analysis combines expert-based and data-driven methods to identify relevant phenomena and explain underlying causalities. The end result is a finite catalog of security-relevant abstract scenarios that can be used in the system development process.
Automation Risks
The automation risk method describes an integrated approach to the safety analysis of specific automated systems based on the identification and quantification of critical scenarios. The method is designed to accompany the entire development process of a system by introducing iterative feedback loops between concept and design phases. To cover both functional safety and SOTIF aspects, it combines established hazard and risk analysis methods and adds adaptations and extensions that enable applicability for safety analysis to automated systems. The approach aims to support existing safety processes in the automotive domain according to ISO 26262 and ISO 21448 standards and to complement them as needed.
