Multi-sensor Navigation and Integrity for UAVs

Vision-aided Navigation using Fiducial Markers for Vertiport Operations of Urban Air Mobility

Motivation

Safety of urban air mobility operations depends critically on high-accuracy, high-integrity navigation. The most demanding phases are at the Final Approach and Take-off (FATO) areas. In these low-altitude zones, relying solely on conventional Global Navigation Satellite Systems (GNSS) is challenging because GNSS performance deteriorates near the ground in urban areas due to severe multipath reflections, poor satellite geometry, and vulnerability to interference. To augment GNSS-based navigation, the German Aerospace Center (DLR) is developing a reliable visual-aided positioning system that employs fiducial markers (e.g. ArUco and AprilTag markers) arranged in a layout compliant with the EASA (European Union Aviation Safety Agency) vertiport-marking guidelines. As the outcome of the research, it is expected to enhance operational safety and provide a robust, integrity-aware navigation solution that can be integrated with existing GNSS-based solutions safe UAM operations in the most critical flight phases at FATO areas.

Landmark-based Visual Navigation at Marker-aided Vertiports

A marker layout that satisfies the EASA vertiport-marking specifications is designed. The markers are geo-referenced before the operation, so that the absolute positions of the markers can be found from a look-up table either stored onboard or sent via U-Space services. During the flight operations, an onboard camera detects the markers in the image stream, and the resulting 2-D pixel coordinates serve as measurements for estimating the pose of the camera (and therefore the drone) relative to the marker frame.

This pose is then transformed into the position in the global reference frame using the pre-known geo-reference of the detected markers. The use of fiducial markers provides the vision system with known signal patterns containing sufficient redundancy, which is necessary to ensure system integrity.

Visual Positioning Algorithm Robust to Initial Pose Error

Vision-based navigation relies on estimating a camera’s 6-DOF pose (position and attitude) using a set of visible features with known locations on the map, a task commonly formulated as the Perspective-n-Point (PnP) problem. However, the performance of the pose estimation is highly sensitive to the initial rough estimate of the camera pose and the feature noise due to the strong nonlinearity of the measurement model. This issue has significantly limited the use of visual navigation in applications that require high reliability and robustness. Innovative algorithms are developed by DLR to exhibits a wide convergence range, enabling precise camera pose estimation even under high measurement noise and large initialization errors. The problem formulation in the algorithm can also be easily applied to validate if it has converged to the global optimum.

Related Publications

  • Crespillo, O. G., Zhu, C., Simonetti, M. et al. (2024). “Vertiport navigation requirements and multisensor architecture considerations for urban air mobility,” CEAS Aeronautical Journal, Jul. 2024.
  • Triolo, A., Zhu, C., Meurer, M. (2024). "Reliable Camera-Based Positioning Robust to Initial Pose Error," Proceedings of the 2024 International Technical Meeting of the Institute of Navigation, Long Beach, California, January 2024, pp. 546-560.

Towards the Integrity of Vision-based Navigation

Motivation

Vision-based navigation can be used for sensor fusion or even as a standalone solution in GNSS-denied environments. However, the lack of integrity protections has constrained its utilization in safety-critical applications. To address integrity, DLR adopts a multi-layer system architecture that isolates the distinct error sources inherent to landmark-based visual navigation, such as feature detection error, marker-placement error, camera-calibration error, and algorithm convergence error, etc. The quantification of different errors and their impact on positioning integrity serve as the foundation towards future vision-based automated navigation for applications with certification requirements.

Multi-domain Fault Detection and Exclusion Framework

Since extracting geometric pose information from raw image measurements requires a series of sophisticated processing steps, several different error sources in the whole processing chain can have a significant impact on the final position error. In order to achieve a quantifiable integrity metric, we proposed a multi-domain fault detection and exclusion framework for visual navigation. All the critical error sources in the processing chain should be monitored and the integrity monitors can be applied in the domain where the error has best observability and distinguishability from other errors.

Nonlinearity Error and Certifiability of Global Optimum

Since the visual navigation measurement equation is highly nonlinear, the pose estimation problem may have multiple local optima. These local optima can have residuals similar to the global optimum while producing large position errors, potentially causing integrity failures. We propose an analytical validation method to determine whether a solution is globally optimal. This method provides a means of monitoring convergence errors and thereby supports the development of certifiable camera-based navigation. In addition, we develop a nominal error propagation pipeline based on Lie-group parameterization to reliably bound propagated errors under strong nonlinearities.

Related Publications

  • Zhu, C., Meurer, M., Günther, C. (2022). Integrity of visual navigation—developments, challenges, and prospects. NAVIGATION: Journal of the Institute of Navigation, 69(2).
  • Lee, Y.-H. and Zhu, C. (2024). Study on the uncertainty propagation of feature-based visual navigation for critical flight phases of urban air mobility. ION GNSS+ 2024 , Baltimore, Maryland, September 2024.
  • Triolo, A., Zhu, C., Meurer, M. (2024). "Certifiability Analysis of the Global Optimality in Camera-Based Positioning with SEC-PnP Algorithm," ION GNSS+ 2024, Baltimore, Maryland, September 2024.

Local GNSS Augmentation for Urban Air Mobility: U-GBAS

Motivation

GNSS standalone does not meet the accuracy requirements for Urban Air Mobility. The Ground Based Augmentation System (GBAS) is an established means of providing accuracy and integrity in aviation, but its hardware is not suitable for UAM integration. UAM operations — take-off and landing at vertiports as well as safe separation en route — demand reliable, integrity-capable positioning, while urban environments degrade GNSS through multipath and limited satellite visibility. A cost-effective, scalable augmentation approach is therefore needed to enable the safe integration of UAM into future urban airspace.

Concept

U-GBAS is a concept for local GNSS augmentation derived from the established GBAS in civil aviation. It provides reliable and accurate GNSS measurements for integration into a redundant, safe and reliable integrated navigation architecture tailored to Urban Air Mobility (UAM) — in particular for take-off and landing operations at vertiports and for safe separation of UAM vehicles en route. By using less expensive hardware than traditional GBAS, the concept makes integration into future urban airspace easier and more cost-effective, both in terms of ground infrastructure demands and on-board navigation hardware. The cornerstone of the concept is its integration with other navigation methods (e.g. INS, barometric sensors, cameras), yielding a multisensory solution that includes integrity information. Preliminary UAV flight trials in realistic urban scenarios (flights near and between office buildings and near a major bridge in Hamburg) as well as long-term rooftop measurements yielded promising initial results.

Hardware Prototype

Currently under development: Improved version of the U-GBAS ground station prototype featuring:

  • Distributed internal (pre-)processing capability (Raspberry-Pi based)
  • Multi-frequency Multi-Constellation GNSS receiver (L1/L5, GPS+GAL+BDS)
  • Connectivity to central processing facility/U-Space service over LTE, Wifi or Ethernet
  • Real-time monitoring (local and remote)
  • Optionally battery/solar-powered for remote-operations
  • Optional integration of barometric sensors to support vertical conversion service

Related Publications

  • Gerbeth, D., Caamano, M., & Zhu, C. (2023). Local Differential GNSS Augmentation for Integration into Urban Air Mobility. Engineering Proceedings, 54(1), 40. https://doi.org/10.3390/ENC2023-15442

Barometric Vertical Navigation in Urban Air Mobility and Vertiport Contexts

Motivation

Urban Air Mobility (UAM) intends to provide high‑frequency, low‑altitude transport services (air‑taxis, delivery UAVs, etc.) within densely built‑up environments. The safety of such operations hinges on the availability of a precise and reliable vertical navigation solution during the most critical phases of flight, i.e. vertical take‑off and landing (VTOL) operations and the subsequent low‑altitude manoeuvres in the vicinity of vertiports.

In dense city environments the Global Navigation Satellite System (GNSS) that underpins most current navigation solutions is subject to several well‑documented limitations. Due to the urban environment, multi-path and signal interference GNSS solution can suffer in vertical accuracy, availability, and integrity. A complementary, highly available source of altitude information that is independent of GNSS is therefore required

Ground-Augmented Barometric System for Vertiport Operation

Barometric altimeters determine altitude by measuring the ambient atmospheric pressure, a principle that is inherently independent of GNSS and immune to radio‑frequency interference. The low mass, modest cost, and ease of integration of barometric sensors make them an attractive, highly scalable solution for any Urban Air Mobility (UAM) platform.

The German Aerospace Center (DLR) is actively researching the use of barometric sensors as a dedicated vertical navigation aid for Urban Air Mobility (UAM) at vertiports. The system consists of a ground station at the vertiport and an airborne barometer. By fusing ground‑based reference data with onboard pressure measurements, the system can deliver precise altitude information even when GNSS signals are unavailable or degraded. This approach promises to bridge critical flight phases in GNSS‑denied airspace.

Methodology

The ground station continuously provides local air pressure and temperature, timestamps the data and transmits it in real time to air vehicles in the vicinity of the vertiport.

The airborne system compares its own pressure readings with the ground reference, applies a Kalman filter to bound measurement uncertainties, and converts the resulting geopotential height into a geodetic altitude.

This way, the ground‑augmented barometric vertical navigation system provides a GNSS‑independent, highly available vertical reference. It also facilitates the harmonization of altitudes and establishment a common altitude reference system (CARS).

Related Publications

  • M. Simonetti and O. G. Crespillo, “Robust Modeling of Geodetic Altitude from Barometric Altimeter and Weather Data,” in ION GNSS+, The International Technical Meeting of the Satellite Division of The Institute of Navigation, ser. GNSS 2021. Institute of Navigation, Oct. 2021.
  • S. Heilein and O. G. Crespillo, “Ground-Corrected Barometric Vertical Navigation System for Vertiport Operations,” in 2024 AIAA DATC/IEEE43rd Digital Avionics Systems Conference (DASC), 2024, pp. 1–10.
  • O. G. Crespillo, C. Zhu, M. Simonetti et al., “Vertiport navigation requirements and multisensor architecture considerations for urban air mobility,” CEAS Aeronautical Journal, Jul. 2024.
  • M. Simonetti and O. G. Crespillo, “Geodetic altitude from barometer and weather data for GNSS integrity monitoring in aviation,” NAVIGATION: Journal of the Institute of Navigation, vol. 71, no. 2, 2024.