Mixed Reality Technologies in aviation
Mixed reality technologies are opening up new possibilities for realistic, flexible and cost-efficient pilot training. At the Institute for Flight Systems, simulation and visualisation systems are being developed for both manned and unmanned aircraft. The goal is a safer, more sustainable air transport system for the future.
Flight simulation with mixed reality technology
Future air traffic concepts in the context of urban and suburban air mobility rely on the use of highly automated small aircraft with innovative propulsion systems. To ensure that qualified pilots are always available to operate these aircraft, modern, practice-oriented training concepts are being developed that heavily rely on flight simulators.
Virtual reality (VR) and mixed reality technologies (MR) offer promising, forward-looking solutions in this field. They enable cost-effective training with a high level of immersion – that is, a realistic experience of being fully immersed in a simulated environment. For these novel technologies to be used in pilot training, the training must take place on devices certified by aviation authorities.
Furthermore, the Institute of Flight Systems is working with actors from industry and training organisations to develop innovative simulation systems for the field of Advanced Air Mobility (AAM). The goal is to create realistic, flexible and sustainable pilot training.
These technologies are also being used in aircraft development – for example in the S²TOL (a short take-off capable autogyro) and the FGAA (a four-seater electric aircraft). Their flight mechanics models are integrated into the virtual environment, enabling realistic testing in a digital space.
Integration of Augmented, Virtual and Mixed Reality in Future Military Aircraft
In military conflicts the use of unmanned systems and sensors is increasing, leading to a greater amount of data that the crew in mission aircraft's cockpits has to manage. To enable the aircrews to carry out their missions with optimal situational awareness, research is conducted on the design of cockpits, including the integration of helmet displays with augmented, virtual and mixed reality.
The technologies are tested in flight simulators by utilizing professional wargaming software, in order to assess their suitability for deployment in future conflicts. This research is conducted in close cooperation with the Bundeswehr and focuses not only on the use of helmet displays for pilots, but also on their use for weapon system officers and loadmasters of transport aircraft.
Development of Innovative Visualisation and MR/VR Software Solutions
Clear and intuitive visualisation of simulation models is essential for understanding complex technical systems. In aviation, the depiction of cockpit instruments and external views plays a key role in creating realistic simulation experiences and credible immersion.
To meet these demands, DLR has developed its own high-performance visualisation software: DLR SimVis. This tool enables real-time rendering of simulation data such as from Simulink or Modelica and allows content from various sources to be integrated within a single user interface.
DLR SimVis is used in both high-dynamic motion simulators and aboard research aircraft. It not only provides a full virtual environment, but also delivers additional information to the flight crew, for example via head-up display functionality.
The software framework is continuously enhanced to meet evolving requirements, especially for Virtual Reality (VR) and Mixed Reality (MR) applications. DLR SimVis is not limited to airborne systems; it is also used in ground vehicles, robotic platforms, satellites, and launch systems.
Mixed reality simulation for rotorcraft
At the Institute of Flight Systems Technology is researching a new technology: mixed reality glasses (MR) in helicopter simulation. To meet the challenges of an increasing pilot demand and improve helicopter flight safety, DLR and its project partners are looking for more efficient and cost-effective training options. MR technology has the potential to complement traditional simulators by providing an affordable, mobile training platform and a safe training environment for helicopter pilots.
Mixed Reality in Unmanned Flight Operations
In the field of unmanned aviation, the use of mixed reality offers significant advantages, which are being actively explored at the Institute. In particular, the application of MR both in the training of drone pilots and in ongoing flight operations by safety pilots are noteworthy.
In training, realistic scenarios can be simulated without the need of an actual aircraft, allowing for risk-free practice under near-real conditions. During operations, real-time data can be visualized in the safety pilot’s headset. This enables the pilot to take control of the aircraft at any time if necessary, ensuring a safe flight even beyond visual line of sight (BVLOS).
Current research challenges include ensuring a secure and stable connection for data transmission – especially for long distances – as well as improving the acceptance and usability of MR systems for safety pilots, and supporting compliance with and the development of regulatory frameworks.
