2. February 2021
Maritime helicopter missions

Aug­ment­ed re­al­i­ty as­sists he­li­copter pi­lots with de­mand­ing flight ma­noeu­vres at sea

More:
Aeronautics
DLR research helicopter EC135 FHS on landing approach
DLR re­search he­li­copter EC135 FHS on land­ing ap­proach
Image 1/5, Credit: DLR (CC-BY 3.0)

DLR research helicopter EC135 FHS on landing approach

DLR has test­ed com­mer­cial aug­ment­ed re­al­i­ty glass­es in flight for the first time. One of the chal­lenges is en­sur­ing that the head-track­ing sys­tem of the glass­es can cor­rect­ly dis­tin­guish the move­ments of a pi­lot’s head from the mo­tion of the he­li­copter. DLR re­searchers have de­vel­oped spe­cial soft­ware that can do ex­act­ly this. The soft­ware in­cor­po­rates this ca­pa­bil­i­ty in­to the dis­play sys­tem to in­di­cate ob­sta­cles.
HEDELA – augmented reality glasses in use
HEDELA – aug­ment­ed re­al­i­ty glass­es in use
Image 2/5, Credit: DLR (CC-BY 3.0)

HEDELA – augmented reality glasses in use

An ini­tial test flight with aug­ment­ed re­al­i­ty glass­es. In ad­di­tion to an ar­ti­fi­cial hori­zon (red line), the he­li­copter pi­lot is pro­vid­ed with key in­for­ma­tion from the in­stru­ment dis­play with­in the cock­pit, di­rect­ly in his im­me­di­ate field of vi­sion. This in­cludes the first lim­it in­di­ca­tor (FLI), which dis­plays the torque, tur­bine speed and tur­bine tem­per­a­ture at the same time.
Test pilot wearing augmented reality glasses
Test pi­lot wear­ing aug­ment­ed re­al­i­ty glass­es
Image 3/5, Credit: © DLR. All rights reserved

Test pilot wearing augmented reality glasses

DLR test pi­lot Se­bas­tian Soffn­er wear­ing the Mi­crosoft HoloLens 2 aug­ment­ed re­al­i­ty glass­es in­side the DLR re­search he­li­copter Ac­tive Con­trol Tech­nol­o­gy / Fly­ing He­li­copter Sim­u­la­tor (ACT/FHS). The vi­su­al as­sis­tance sys­tem is de­signed to help pi­lots per­form com­plex flight ma­noeu­vres. The pi­lot is shown im­por­tant in­for­ma­tion about the cur­rent flight sta­tus and any ob­sta­cles in his im­me­di­ate field of vi­sion.
HEDELA flight system technology for a high-sea ship deck landing
HEDELA flight sys­tem tech­nol­o­gy for a high-sea ship deck land­ing
Image 4/5, Credit: DLR (CC-BY 3.0)

HEDELA flight system technology for a high-sea ship deck landing

Land­ing on the deck of a ship out at sea is a ma­jor chal­lenge even for ex­pe­ri­enced pi­lots. Wake tur­bu­lence forms im­me­di­ate­ly to the rear of the ship’s deck and makes it dif­fi­cult for a he­li­copter to land.
Professor Stefan Levedag
Pro­fes­sor Ste­fan Levedag
Image 5/5, Credit: © DLR. All rights reserved

Professor Stefan Levedag

Pro­fes­sor Ste­fan Levedag is the Di­rec­tor of the DLR In­sti­tute of Flight Sys­tems.
  • As part of the HEDELA project, DLR and the aviation service of the German Federal Police are researching assistance systems to facilitate helicopter landings on ship decks.
  • Augmented reality glasses take the pressure off of pilots in poor visibility conditions, superimposing landmarks for orientation and other important information directly onto their field of vision.
  • Successful flight tests have been carried out with DLR's FHS research helicopter.
  • Focus: Aviation, security, helicopter research

Whether they are facing fog, rain or darkness, a high-sea landing on a ship deck under poor visibility conditions is a major challenge, even for experienced helicopter pilots. As part of the Helicopter Deck Landing Assistance (HEDELA) project, the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR) is working with the aviation service of the German Federal Police to develop flight assistance systems that help pilots land safely and accurately on ships. Augmented reality glasses have now been used for the first time in flight tests with DLR’s FHS research helicopter.

Helicopters play an important role in rapidly expanding offshore industry. Their ability to take off and land vertically, hover in one spot and reach high flight speeds means that they are well suited for a variety of applications at sea. Problems arise, however, under poor visibility conditions, often caused by sea fog, precipitation and darkness. "Out at sea, pilots lack fixed reference points for orienting themselves," says Stefan Levedag of the DLR Institute of Flight Systems. "All they see around them is the open sea, and the moving ship offers a relatively small space on which to land." This is compounded by the wake turbulence on the ship's deck, which can make it even more difficult for the helicopter to safely set down.

Visual assistance system helps pilots during landing approach

As part of the HEDELA project, DLR has developed and validated special displays for landings on ship decks. The aim is to support pilots throughout the entire flight, including providing obstacle-free flight path planning (trajectories) for approach, transfer and takeoff. Augmented reality (AR) glasses take the pressure off pilots performing these tasks in poor visibility conditions, superimposing landmarks for orientation and other important information directly onto their field of vision. In thick cloud, for instance, the floating ship is outlined in bright green. Important information about flight speed, altitude and position, plus an artificial horizon are also made visible within the pilot's field of vision. "The superimposed symbols are incredibly beneficial for helping pilots find their bearings," says Project Manager Malte-Jörn Maibach. "The major benefit is that the pilot doesn’t have to be constantly switching his gaze between the instruments in the helicopter cockpit and the view outside. That allows him to keep his eyes on the landing point." This takes the pressure off the pilot and increases his situational awareness. In doing so, it also helps to optimise the operational capability and safety of maritime helicopter missions.

Current helicopter cockpit developments focus on reducing and combining display devices. One example is the first limit indicator (FLI), which presents the most important data relating to the status of the propulsion system (torque, turbine speed and turbine outlet temperature) using a single monitoring instrument. This type of flight assistance system, which was also implemented in the tested head-mounted display (HMD) described below, can significantly improve the general flight characteristics and handling properties of a helicopter.

Look­ing through aug­ment­ed re­al­i­ty glass­es dur­ing an in-flight sys­tem test
The holo­graph­ic dis­play re­mains aligned with the world even dur­ing dy­nam­ic ma­noeu­vres.
Credit: DLR (CC-BY 3.0)

The researchers developed the design of the display system during a predecessor project, Helicopter Flight Safety in Maritime Operations (HELMA), and tested it successfully using a simulator. DLR has now trialled the use of commercial augmented reality glasses in flight for the first time during tests with its Active Control Technology / Flying Helicopter Simulator (ACT/FHS) research helicopter, a modified EC135. HMDs – helmets with integrated displays worn by pilots – are already widely used in the military sector and offer a number of benefits. However, such systems are very expensive. As part of their work, the researchers are therefore drawing upon commercial systems and developing them with the specific requirements of maritime helicopter missions in mind.

"The advantage is that commercial systems are cheaper to procure, while still making use of very advanced technology," says Maibach. The head-tracking feature of AR glasses presents a notorious challenge, however, as it is not designed for use in moving systems, such as by the driver of a vehicle. "The system must be able to distinguish the movements of a pilot's head from the motion of the helicopter, which causes the outside world to appear to move," says Aerospace Engineer Christian Walko of the Institute of Flight Systems. For this purpose, DLR researchers have developed software that recognises the position of the pilot’s head in relation to the moving helicopter and takes it into account when displaying the holographic symbols. The researchers are currently analysing data from the flight tests. They are also investigating how the AR glasses performed in the air in conjunction with the newly developed head-tracking algorithm and attempting to determine the impact of helicopter vibrations on the entire system configuration.

New display systems are of considerable interest to helicopter pilots, who have to perform highly complex flight manoeuvres that require them to keep the helicopter under control while maintaining a constant awareness of the mechanical flight parameters. Innovative displays make it possible to integrate the relevant information with standard interfaces and help make staying within these limits as intuitive as possible.

The challenge of landing on a ship deck – three questions for the Director of the DLR Institute of Flight Systems, Professor Stefan Levedag

Why is the DLR Institute of Flight Systems conducting research into helicopter landings on ship decks?

Landing on a ship deck is one of the most challenging flight manoeuvres for a helicopter pilot. Besides having to cope with a moving landing platform, they are often beset by strong winds, turbulence caused by the contours of the ship, a lack of reference points for gauging position and speed, and poor visibility with no horizon. Providing pilots with an optimal system for coping with such extreme conditions is a major challenge, but DLR researchers are using cutting-edge technology to develop effective solutions.

Such assistance systems are already used in the military sector. Why are there are not yet any affordable alternatives for civil aviation? When might such a system become available for civilian operators?

Military systems are subject to very stringent additional requirements, such as compatibility with night vision technology, and are therefore more costly. Integrating new features is more feasible in that type of environment than with a commercial product, for which there is a pressure to keep costs as low as possible.

New systems, such as the HoloLens, also have to be qualified for future use in aircraft. They have to be able to withstand heavy vibrations, temperature differences and electromagnetic fields, for instance, without compromising on accuracy. Major obstacles often arise when attempting to deploy innovative solutions from the field of consumer electronics, and overcoming them takes time and considerable resources. As a result, it is difficult to predict when such technology might successfully transition into an aviation product. The hurdles are particularly high given the difficult period that the aviation sector is currently experiencing.

What is special about the ACT/FHS research helicopter?

The EC135 ACT/FHS helicopter is a unique instrument for research. It has open interfaces that make it possible to feed all of the essential flight status data into experimental systems, including new displays or even visualisations such as the HoloLens. Due to the exceptional architecture of the system, these new elements do not have to be approved, as would be necessary with a commercial application. This even applies to new systems and functions that feed control commands into the helicopter.

Security research at DLR

DLR's security research and development activities are planned and monitored in conjunction with partners in government, academia, industry and international organisations. As such, the interdisciplinary field of security research combines the core strengths of established DLR programmes in the fields of aviation, space, energy and transport. In total, over 20 DLR institutes and facilities are contributing to the development, testing and evaluation of technologies, systems and concepts within their security-related work, and are also conducting analysis and assessments of security-related applications.

Contact
  • Jasmin Begli
    Cor­po­rate Com­mu­ni­ca­tions Braun­schweig, Cochst­edt, Stade and Trauen
    Ger­man Aerospace Cen­ter (DLR)

    Pub­lic Af­fairs and Com­mu­ni­ca­tions
    Telephone: +49 531 295-2108
    Fax: +49 531 295-12100
    Lilienthalplatz 7
    38108 Braunschweig
    Contact
  • Malte-Jörn Maibach
    Ger­man Aerospace Cen­ter (DLR)
    In­sti­tute of Flight Sys­tems
    Ro­tor­craft
    Lilienthalplatz 7
    38108 Braunschweig
    Contact
  • Stefan Levedag
    Di­rec­tor
    Ger­man Aerospace Cen­ter (DLR)
    In­sti­tute of Flight Sys­tems
    Telephone: +49 531 295-2600
    Lilienthalplatz 7
    38108 Braunschweig
    Contact
Newsletter

Newslet­ter

Stay up to date and sub­scribe to the DLR newslet­ter with ar­ti­cles from the DLR ed­i­to­ri­al team in Ger­man and En­glish.

Cookies help us to provide our services. By using our website you agree that we can use cookies. Read more about our Privacy Policy and visit the following link: Privacy Policy

Main menu