Improved safety and security through Alignment with Reality

In recent years, digital twins have become a key technology for the resilience of critical infrastructure. Linking virtual models to real-time data enables infrastructure operators to continuously monitor the condition of their facilities, simulate crisis scenarios, and make informed decisions. However, for this technology to function reliably, the digital replica must accurately reflect the actual condition of the real-world facility at all times.

In practice, however, maintenance work, environmental factors, and unexpected events continuously lead to changes. As a result, the digital model and reality quickly diverge from one another. To ensure the long-term quality of digital twins, it is therefore crucial to detect such deviations early on. This is particularly important in safety- and security-critical areas such as rail transport.

To address this challenge, Tamara Lenhard and Tobias Koch from the DLR Institute for the Protection of Terrestrial Infrastructures presented a new, innovative approach at the UAS Rail 2026 conference hosted by the German Center for Rail Transport Research. The method is capable of automatically detecting such discrepancies and the developed prototype compares real-world image data with synthetic views of the digital twin. It analyzes potential differences using modern foundation models—current AI models that have been trained on large datasets and therefore better understand complex image content.

To do this, real and synthetic scenes are first converted into semantic maps. These represent objects and infrastructure elements such as roads, railways, bridges, buildings, or vegetation in a structured manner. Vision Transformer models then analyze the generated representations and compare their semantic features to identify potential discrepancies between reality and the digital twin.

To further evaluate the detected anomalies, vision-language models are used, which combine image information with natural language. The model describes the respective image regions in text form and checks whether the content of reality and the digital twin match. This additional analysis enables more robust detection of relevant changes and specifically reduces misclassifications and purely visual artifacts. Initial investigations in a controlled environment are already showing promising results.

A particular advantage of the presented approach is that it is highly flexible and works independently of the specific industry. The method can be adapted to other application areas, such as rail transportation, with relatively little effort. To do so, only the domain-specific objects and infrastructure elements—such as tracks, switches, or signals—need to be adjusted, while the rest of the process remains unchanged. The approach is also flexible with regard to the sensor platform used and works with both stationary cameras and mobile platforms such as inspection vehicles or drones.