AcListant®

For human beings it is taken for granted, for a technical system not implemented so far: the possibility to listen to the conversation of other people and derive activities from the understood contents. The project AcListant® (Active Listening Assistant, 2013–2015) implemented a prototype for understanding the controller-pilot-communication.

The successor project AcListant®-Strips, started in 2015, uses Assistant Based Speech Recognition (ABSR) as an additional input device to replace mouse and keyboard inputs. The goal is to reduce controller workload and to increase aircraft throughput.

© DLR. All rights reserved.

AcListant®-Strips

Although the expected results were achieved by the AcListant® project, i.e. improving AMAN performance, the stakeholders see the main benefits when Assistant Based Speech Recognition (ABSR)  is used as an additional input device to replace mouse and keyboard inputs into the Flight Processing Data System (FPDS).

In May 2015, therefore, the follow-up project AcListant-Strips was started. Paper flight strips have been or will be replaced, also in approach control, by most major ANSPs during the next years. This will, however, increase controller workload, if the controllers are forced to maintain the electronic flight strips/radar labels by mouse and keyboard. This will probably also have negative impact on aircraft throughput. AcListant-Strips can be a representation of an electronic flight strip concept even minimizing the input data maintenance actions the controller has to perform.

The AcListant-Strips project quantifies the benefits, if assistant based speech recognition is used as primary input device for label maintenance. Mouse and keyboard inputs are only necessary in the remaining cases, where speech recognition fails.

References

  1. Motivation of AcListant project: H. Helmke, H. Ehr, M. Kleinert, F. Faubel, and D. Klakow, “Increased Acceptance of Controller Assistance by Automatic Speech Recognition,” in: “Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013)”, Chicago, IL, USA, 2013
  2. Result of AcListant-Trials in October 2014: H. Helmke et al. Assistant-Based Speech Recognition for ATM Applications, in: “Eleventh USA/ Europe Air Traffic Management Research and Development Seminar (ATM2015)”, Lisbon, Portugal, 2015.
  3. Result of AcListant- Final Validation Trials in February and March 2015: Hejar Gürlük, Hartmut Helmke, Matthias Wies, Heiko Ehr, Matthias Kleinert, Thorsten Mühlhausen, Kathleen Muth, Oliver Ohneiser: Assistant Based Speech Recognition – Another Pair of Eyes for the Arrival Manager, 34th DASC, Sep. 2015, Prague, Czech Republic
  4. Assistant-Based Speech Recognition: Youssef Oualil, Marc Schulder, Hartmut Helmke, Anna Schmidt, Dietrich Klakow: Real-Time Integration of Dynamic Context Information for Improving Automatic Speech Recognition, Interspeech 2015, Sep. 2015, Dresden, Germany
  5. Using context for Speech Recognition: Anna Schmidt, Youssef Oualil, Oliver Ohneiser, Matthias Kleinert, Marc Schulder, Arif Khan, Hartmut Helmke, Dietrich Klakow: Context-Based Recognition Network Adaptation for Improving On-Line ASR in Air Traffic Control, Proceedings of 2014 IEEE Spoken Language Technology Workshop (SLT 2014), South Lake Tahoe, Nevada, USA, Dec 7-10, 2014.
  6. HMI used in Validation Trials: Oliver Ohneiser, Hartmut Helmke, Heiko Ehr, Hejar Gürlük, Michael Hössl, Matthias Kleinert, Thorsten Mühlhausen, Maria Uebbing-Rumke, Youssef Oualil, Marc Schulder, Anna Schmidt, Arif Khan, Dietrich Klakow: Air Traffic Controller Support by Speech Recognition, in Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics AHFE 2014, Kraków, Poland, 19-23 July 2014. Edited by T. Ahram, W. Karwowski and T. Marek.
  7. Validation design: Maik Friedrich, Hartmut Helmke, Thorsten Mühlhausen, Oliver Ohneiser, Heiko Ehr, Heiko  (2015) Combined metrics for the evaluation of air traffic controller assistance systems.   In: Proceeding of the 6th International Conference on Applied Human Factors and Ergonomics 2015. Springer Press. 6th International Conference on Applied Human Factors and Ergonomics, 27. Jul. – 31. Jul. 2015, Las Vegas, USA.

Media Coverage

  1. audio pod-cast Deutschlandfunk (in German) (2015-08-15): Der Slang-Versteher: Neues Assistenz-System für Fluglotsen
  2. Saarländischer Rundfunk (in German 2015-07-28) Sprachforscher machen das Fliegen sicherer
  3. IT-daily, 2015-07-28: Sprachforscher machen das Fliegen sicherer: Assistenzsystem versteht Gespräche von Lotse und Pilot
  4. DFS activities for speech recognition: Jörg Buxbaum, Hon. Hartmut Helmke, Oliver Rühl, Michael Slotty: Speech Recognition in Air Traffic Control Context (German Title: Spracherkennung im Flugsicherungsumfeld), In „Innovation im Fokus 1, 2015“, publication of DFS, pp. 23-29
  5. DFS Reseach: Christopher Belz: Forschen für Flugsicherung von morgen, In „Transmission 1/2014“, publication of DFS, pp. 24-25
  6. Saarbrücker Zeitung (in German, 2014-02-04): Neue Software der Saar-Universität soll den Flugverkehr sicherer machen
  7. audio post-cast SWR/NDR (2013-11-26): Innovation im Flugverkehr: Lotsenassistenzsystem mit Spracherkennung
Scroll to Top