Accéder directement au contenu Accéder directement à la navigation
Article dans une revue

Deep Reinforcement Learning based Path Stretch Vector Resolution in Dense Traffic with Uncertainties

Abstract : With the continuous growth in the air transportation demand, air traffic controllers will have to handle increased traffic and consequently, more potential conflicts. This gives rise to the need for conflict resolution advisory tools that can perform well in high-density traffic scenarios given a noisy environment. Unlike model-based approaches, learning-based approaches can take advantage of historical traffic data and flexibly encapsulate environmental uncertainty. In this study, we propose a reinforcement learning approach that is capable of resolving conflicts, in the presence of traffic and inherent uncertainties in conflict resolution maneuvers, without the need for prior knowledge about a set of rules mapping from conflict scenarios to expected actions. The conflict resolution task is formulated as a decision-making problem in a large and complex action space. The research also includes the development of a learning environment, scenario state representation, reward function, and a reinforcement learning algorithm inspired from Q-learning and Deep Deterministic Policy Gradient algorithms. The proposed algorithm, with two stages decision-making process, is used to train an agent that can serves as an advisory tool for air traffic controllers in resolving air traffic conflicts where it can learn from historical data by evolving overtime. Our findings show that the proposed model gives the agent the capability to suggest high quality conflict resolutions under different environmental conditions. It outperforms two baseline algorithms. The trained model has high performance under low uncertainty level (success rate ≥ 95% ) and medium uncertainty level (success rate ≥ 87%) with high traffic density. The detailed analysis of different impact factors such as environment’s uncertainty and traffic density on learning performance are investigated and discussed. The environment’s uncertainty is the most important factor which affects the performance. Moreover, the combination of high-density traffic and high uncertainty will be the challenge for any learning models.
Type de document :
Article dans une revue
Liste complète des métadonnées

https://hal-enac.archives-ouvertes.fr/hal-03425669
Contributeur : daniel delahaye Connectez-vous pour contacter le contributeur
Soumis le : jeudi 11 novembre 2021 - 07:26:22
Dernière modification le : mercredi 17 novembre 2021 - 03:55:58
Archivage à long terme le : : samedi 12 février 2022 - 18:09:49

Fichier

TRC_DRL_Conflict_Resolution.pd...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-03425669, version 1

Collections

Citation

Daniel Delahaye, Duc-Thinh Pham, Phu N Tran, Sameer Alam, Vu Duong. Deep Reinforcement Learning based Path Stretch Vector Resolution in Dense Traffic with Uncertainties. Transportation research. Part C, Emerging technologies, Elsevier, 2021. ⟨hal-03425669⟩

Partager

Métriques

Consultations de la notice

155

Téléchargements de fichiers

161