ReliableAI

As part of the ReliableAI project, a 'safety-by-design' methodology is being developed to ensure the reliable and correct development of AI-based perception components. A key part of this approach involves creating a reference dataset covering a wide range of safety-critical scenarios, which will enable the development of a robust safety case. To enable adaptation to a changing environment, we are also developing a continuous AI update process that maps changes to the AI function while maintaining consistent reliability. This process is triggered by quantifying the uncertainties of the perception function using a monitoring tool.

In order to enable bespoke deployment on the target hardware, the methodology for use under hardware constraints is being refined further. To this end, a toolbox offering optimisation techniques for AI functions in terms of computing power and storage space is being developed. Finally, a systematic analysis of the operational safety of the AI system in the event of attacks aims to combine safety and security. Key contributions here include assessing safety aspects and risks associated with using hardware-based AI, developing protection methods against attacks and transferring safety guarantees to compressed models.

Contribution of the Institute for AI Safety and Security

In addition to providing overall coordination, the Institute is developing a rigorous argument regarding the safety of the system as part of a 'safety-by-design' methodology. This approach ensures the safety of the AI-based perception function throughout its entire lifecycle, from the design stage onwards.
Another key focus is establishing an AI update that will enable the system to adapt to a constantly changing environment without compromising reliability. To this end, various perception functions must be investigated to identify the architectural-level structures crucial for developing such an update without any loss of performance.
To ensure the correct functioning of the AI perception system and respond to changes in environmental conditions, an AI monitoring tool is being developed. This tool evaluates uncertainties in the perception model during operation and triggers the update process based on human-annotated ground truth data where necessary. The tool also uses methods to improve explainability. This makes the predictions easier to understand and enables conclusions to be drawn about the overall system's safety. This strengthens trust in the AI and paves the way for the next stages of development.