Acceleration-based Quadrotor Guidance Under Time Delays Using Deep Reinforcement Learning
Résumé
This paper investigates the use of deep reinforcement learning to act as closed-loop guidance for quadrotors and the ability for such a system to be trained entirely in simulation before being transferred for use on a real quadrotor. It improves upon previous work where velocity-based deep reinforcement learning was used to guide the motion of spacecraft. Here, an acceleration-based closed-loop deep reinforcement learning guidance system is developed and compared to previous work. In addition, state augmentation is included due to dynamics delays present. Simulated results show acceleration-based deep reinforcement learning closed-loop guidance has significant performance benefits compared to velocity-based guidance in previous work, namely: a simpler reward function, less overshoot, and better steady-state error. To evaluate the ability for this system to be used on a real quadrotor, the trained system is deployed to the Paparazzi aircraft simulation software, and is implemented on real flight hardware at École Nationale de l’Aviation Civile for an experimental comparison. Experimental results confirm the simulated results—that acceleration-based deep guidance outperforms velocity-based deep guidance and should therefore be used in future work.