RapTOr: Remote Tower Operation with Augmented Vision Videopanorama Human System Interface
Remote Tower Operation (RTO) describes the goal of remote control of small airports and of movement areas of large airports which are not directly visible from the tower.
The direct far view out of the tower windows should be replaced by an augmented vision video panorama. RapTOr (2005-2007) is the first step on the way to the RTO center for the control of multiple small airports and to the Virtual Tower (ViTo) control center for large airports as long term goal. The concept study „Virtual Tower“ (ViTo, 2002 - 2004) provided the starting point for RapTOr project. The results of RapTOr will be carried on in the project RAiCe (Remote Airport Traffic Control Center).
In 2005 the DLR project RapTOr was started in order to realize an RTO experimental system as extension of the Advanced Surface Movement Guidance and Control System (A-SMGCS) at the Braunschweig research airport. Analysis, cognitive modelling and simulation of the tower work procedures support the design and development of the demonstrator.
Research and development goals
1. Work analysis, modeling and simulation of operator decision making within the RTO augmented video panorama workplace environment in order to support the development process and the investigation of design alternatives.
2. Realization of an RTO – experimental environment and a demonstrator for augmented video panorama based remote airport control, including tests and concept validation with professional tower- / apron controllers.
Realization of the RTO experimental System
The direct view out of the tower windows is of central importance for surface traffic control under the present day working conditions of tower and apron controllers. That is why each attempt of surface movement management without direct view out of tower windows represents a revolution for tower controller working conditions.
|Fixture with four high resolution panorama cameras and pan-tilt zoom (PTZ) camera. Braunschweig tower in the background.|
|180° Videopanorama display system with additional PTZ display (right table insert and pen touch input display (left). |
|Research Airport Braunschweig. Red circle: panorama camera system; red square: Visualisation system; red lines indicate observed camera segments; yellow lines depict fiber-optic ASMGCS data network with thick lines indicating Gbit link for panorama video|
Consequently it is assumed that under the guideline of human centered automation, the reconstruction of the direct far view in future A-SMGCS, in addition to the abstract (birds view) ground movement situation display showing the vehicle positions, will greatly improve the transition process to the new work environment and make it acceptable to the user. The design process for a RTO (and future ViTo) work environment relies on domain knowledge from experts (controllers) of the German Air Traffic Control Organisation (DFS), which provide the input data for the cognitive tower work analysis.
|High resolution wide angle projection of video panorama with PTZ (Yellow frame) and infrared camera display (lower left segment). Superimposed augmented vision data in the approach segment: wind and transponder code with multilateration data.|
Demonstration and Results
The first life demonstration of the real – time high resolution 180° - video panorama reconstruction of Braunschweig research airport was realized at the occasion of the „5th ATM R&D Symposiums“ in Oktober 05 [http://atmsymposium.dlr.de]. It proved the possibility of real time tracking of moving aircraft e.g. during final approach and landing, with the PTZ camera.
Initial results of augmented vision by real time attachment to landing aircraft of transponder data with multilateration position was presented at the ICRAT 2006 conference in Belgrade [http://www.icrat.org].
The Institute of Flight Guidance presented the working console at the International Airshow ILA 2008 in Berlin. http://www.dlr.de/desktopdefault.aspx/tabid-4780/7926_read-12593/
Results of the RapTOr project were presented and discussed at the final workshop together with international experts in April 2008. The presentations and discussions focused on the topics: Virtual tower concept, work and task analysis, formal modeling and simulation of basic airport processes and decision making by using colored Petrinets, video reconstruction of the far view out of the tower windows, augmented vision, automatic movement detection, image processing and object tracking using a pan-tilt zoom camera, validation methodology, verification and initial validation of the video panorama system by means of flight testing, and an overview on the Swedish remote tower project ROT/ART.
The RapTOr results are published in a number of publications. Patents are applied for basic featurs of the RTO systems. The RTO console with augmented vision videopanorama display as human-system interface was on exhibit on several occasions, including the governments open doors day.
The follow-up project RAiCe (Remote Airport traffic control Center) continues the RapTOr project with the goal of develoing and testing a center work environment for the remote control of two small airports.
A-SMGCS: Advanced Surface Movement Guidance and Control System
ATM: Air Traffic Management
RapTOr: Remote Tower Operation Research
RTO: Remote Tower Operation
ViTo: Virtual Tower
Publications "Virtual Tower"/ "Remote Tower Operations" from the Institut of Flight Guidance/ Human Factors branch (since 2002)
Thematic Area 1: "Virtual Tower/ Remote Tower Operation"
[1.1] N. Fürstenau, Virtual Reality for Integration, Proc. 12th Scientific Seminar: The Challenges of Integration", DLR, Inst. of Flight Guidance, 30.-31.Oct. 2002, to be published as DLR-Mitteilung, www.dlr.de/
[1.2] N. Fürstenau, M. Rudolph, M. Schmidt, B. Lorenz, T. Albrecht, On the use of transparent rear projection screens to reduce head – down time in the air – traffic control tower, Proc. Human Performance, Situation Awareness and Automation Technology (HAPSA II) 22. -25.3.04, Daytona Beach / Fl, Lawrence Erlbaum Publishers Inc. (2004), 195 – 200
[1.3] N. Fürstenau, M. Rudolph, M. Schmidt, B. Werther, Virtual Tower, in: " Wettbewerb der Visionen 2001 – 2004", Hrsg. Deutsches Zentrum für Luft-und Raumfahrt (2004) pp.16 – 21
[1.4] N. Fürstenau, Virtual Tower, 5th ATM R&D Symposium (DLR, Eurocontrol, EC), Braunschweig, 11.-13.10. 2005 http://atmsymposium.dlr.de
[1.5] M. Schmidt, M. Rudolph, B. Werther, N. Fürstenau, Remote Airport Tower operation with Augmented Vision Video Panorama HMI, Proc. 2nd Int. Conf. Res. in Air Transportation (ICRAT 2006), Belgrade (2006), pp. 221-229
[1.6] M. Schmidt, M. Rudolph, B. Werther, N. Fürstenau: Development of an Augmented Vision Videopanorama Human-Machine Interface for Remote Airport Tower Operation, Proc. HCII2007 Beijing, Springer Lecture Notes Computer Science 4558 (2007) 1119-1128
[1.7] N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, B. Werther, Augmented Vision Videopanorama System for Remote Tower Operation: Initial Validation, Proc. 6th Eurocontrol Innovative Research Workshop, Bretigny, 4.-6. 12. 2007, 125-132
[1.8] N. Fürstenau, C. Möhlenbrink, M. Rudolph, M. Schmidt, W. Halle: Augmented Vision Videopanorama System for Remote Airport Tower Operation. Proc. 26th Int. Congress of the Aeronautical Sciences, I. Grant (Ed.), Anchorage, Sept. 14-19 2008, ISBN 0-9533991-9-2
[1.9] N-Fürstenau. Simulation of bistable perception with long range correlation using reentrant nonlinear perception-attention-memory coupling. Proc. (Abstracts) 12th Annual Meeting, Association for the Scientific Study of Consciousness, Taipei, 19.-22.6.2008, p.75-76
[1.10] N.Fürstenau, M. Mittendorf. Nonlinear dynamics model for simulation of multistable perception using reentrant perception-attention-memory coubling. Proc. (Abstracts) KogWis 2008, 9. Jahrestagung Gesellschaft für Kognitionswissenschaften, Dresden 28.9.-1.10.2008, p. 57
Thematic Area 2: Cognitive Modeling / Simulation of Perception and Decision Making
[2.1] N. Fürstenau, Nonlinear Dynamics of Belief Updating and Decision Making, Proc. IEEE Conf. Systems, Man and Cybernetics, Hammamet / Tunesia (2002). IEEE Cat. no. 02CH37349C, ISBN 0-7803-7438-X, paper TP 1O2
[2.2] N. Fürstenau, Nichtlineare Dynamik der Hypothesenbildung, DGLR-Bericht 2002-04: "Situation Awareness in der Fahrzeug – und Prozeßführung", ISBN3-932182-29-4, Deutsche Gesellschaft für Luft-und Raumfahrt, Hrsg.: M. Grandt, K.-P. Gärtner, (2002), 199 - 216
[2.3] N. Fürstenau, A nonlinear dynamics model of binocular rivalry, in: "The Logic of Cognitive Systems", Proc. 5th Int. Conf. Cognitive Modeling ICCM 2003, Hrsg.: F. Detje, D. Dörner, H. Schaub, Bamberg 10.-12.4.03, pp. 111-116
[2.4] N. Fürstenau, A nonlinear dynamics model of binocular rivalry and cognitive multistability, Proc. IEEE-SMC 2003, IEEE Int. Conf. Systems, Man, and Cybernetics, Washington/DC, (2003), ISBN 0-7803-7953-5, IEEE Cat. no. 03CH37483C, pp. 1081-1088
[2.5] N. Fürstenau, A recursive attention – perception chaotic attractor model of cognitive multistability, Proc. 6th Int. Conf. on Cognitive Modeling (ICCM 2004), Pittsburg/PA July 30 – Aug.1 (2004), M. Lovett, C. Schunn, C. Lebiere, P. Munro, Editors. Lawrence Erlbaum Associates Publishers, Mahwah/NJ (2004), 348 – 349
[2.6] Fürstenau N.: A chaotic attractor model of cognitive multistability, Proceedings of the IEEE 2004 Int. Conf. on Systems, Man and Cybernetics, ISBN 0-7803-8567-5, IEEE cat. no. 04CH37583C. (2004) pp. 853-859
[2.7] Fürstenau N. (2005): Nonlinear Dynamics Model and Simulation of Spontaneous Perception Switching with Ambiguous Visual Stimuli, Proceedings Conf. Human Computer Interaction 2005: Augmented Cognition I, Las Vegas 22.7. – 27.7. 2005, vol. 11: Foundations of Augmented Cognition, paper 109
[2.8] N. Fürstenau, Modellierung und Simulation spontaner Wahrnehmungswechsel bei mehrdeutigen Stimuli in Augmented Vision Systemen, 6. Berliner Werkstatt Mensch-Maschine-Systeme 13.-15.10. 2005, in: VDI – Fortschrittsberichte Reihe 22: Zustandserkennung und Systemgestaltung, ZMMS Spektrum Bd. 19, VDI-Verlag Düsseldorf (2005) 119-124
[2.9] Werther, B., Uhlmann, H.: Ansatz zur modellbasierten Entwicklung eines Lotsenarbeits-platzes. In: Zustandserkennung und Systemgestaltung, Fortschritt Berichte VDI, vol. 22, (2005) pp. 291-294
[2.10] Werther, B., Schnieder, E.: Formal Cognitive Resource Model: Modeling of human behavior in complex work environments. In: Proc. Int. Conf. Computational Intelligence for Modelling, Control & Automation, CIMCA (2005), Wien, pp. 606 – 611.
[2.11] Werther, B.: Kognitive Modellierung mit farbigen Petrinetzen zur Analyse menschlichen Verhaltens, PhD Dissertation, DLR-Inst. of Flight Guidance and Technische Universität Braunschweig, 2006.
[2.12] N. Fürstenau, Modelling and Simulation of Spontaneous Perception Switching with Ambiguous Visual Stimuli in Augmented Vision Systems, in: Perception and Interaction Technologies 6, Lecture Notes in Artificial Intelligence (LNAI), Springer, Berlin , Heidelberg (2006), pp.20-31
[2.13] N. Fürstenau, A Computational Model of Bistable Perception-Attention Dynamics with Long Range Correlations, Proc. 30th German Conf. on Artificial Intelligence (KI2007), in: J. Hertzberg, M. Beetz, R. Englert (Eds.): Lecture Notes in Artificial Intelligence (LNAI), Springer 2007, pp. 251-263
[2.14] N. Fürstenau, M. Mittendorf, Simulation of Bistable Perception with Long Range Correlations Using Perception-Attention-Memory Coupling, Proc. 14th Fall Academy "Theory in Cognitive Neuroscience", Wildbad Kreuth 4.-7.11.07, (2007), 12, Abstract
[2.15] B. Werther, C. Möhlenbrink, M. Rudolph: Colored Petri Net Based Formal Airport Control Model for Simulation and Analysis of Airport Control Processes. V.G. Duffy (Ed.): Digital Human Modeling, HCII 2007, LNCS 4561, pp. 1027-1036, Springer Verlag Berlin 2007