Deep Learning-based Global Local Planner for Autonomous Nano-drones
Autonomous unmanned aerial vehicles (UAVs) are increasingly getting smaller and smarter. Their tiny form factor can be a game-changer in many practical applications such as aerial inspection or assistance in natural disaster areas, to name a few.
A common approach in the state-of-the-art (SoA) of autonomous driving systems is represented by the combination of global and local planning . On the one hand, global planning sees “the whole picture” (for example ensuring the robot is passing through pre-defined check-points given as GPS coordinates), while the local planning takes care of close-proximity maneuvers (e.g., avoiding obstacles).
In this context, the project aims at enabling such a planning paradigm aboard small flying robots (i.e., nano-drones with 10cm in diameter and a few tens of grams in weight). This class of vehicles is particularly appealing but also challenging given all their constraints, e.g., form-factor, payload, power envelope. For this ambitious goal, we envision leveraging a “computationally cheap” deep neural network to act as the global planner , ensuring high-level semantic information is captured from the environment and a more precise local planner based on a novel time-of-flight (ToF) sensor  that output small-size (8x8) depth maps.
A key aspect of this project is represented by the deployment of all sensors and computational modules aboard a Crazyflie 2.1 nano-drone , equipped with a powerful AI-deck  companion board that enables the onboard execution of multiple workloads thanks to a parallel ultra-low power octa-core (PULP) GAP8 System-on-Chip .
- Familiarity with C programming.
- Knowledge of basic python programming and deep learning frameworks (e.g., PyTorch)
- Basic knowledge of parallel programming and Host/Accelerator paradigm, or willing to learn
- Basic knowledge/experience in embedded programming favorable
- 15% Literature and theory study, familarization with existing code
- 30% Python programming, convolutional neural network training
- 30% C embedded programming (STM MCU and PULP programming/optimization)
- 15% In-field test and verification
- 10% Report writing
Detailed Task Description
Meetings & Presentations
Weekly meetings will be held between the student and the assistants. The exact time and location of these meetings will be determined within the first week of the project. These meetings will be used to evaluate the status and progress of the project. Besides these regular meetings, additional meetings can be organized to address urgent issues as well.
-  D. Palossi et al., "An energy-efficient parallel algorithm for real-time near-optimal UAV path planning," in Proceedings of the ACM International Conference on Computing Frontiers, pp. 392-397. 2016. https://dl.acm.org/doi/abs/10.1145/2903150.2911712
-  D. Palossi et al., "A 64-mW DNN-based visual navigation engine for autonomous nano-drones," in IEEE Internet of Things Journal, 6(5), 8357-8371, 2019. https://ieeexplore.ieee.org/abstract/document/8715489
-  STM time-of-flight (ToF) matrix https://www.st.com/en/imaging-and-photonics-solutions/vl53l5cx.html
-  Bitcraze Crazyflie 2.1 https://www.bitcraze.io/products/crazyflie-2-1/
-  Bitcraze AI-deck 1.1 https://store.bitcraze.io/products/ai-deck-1-1
-  PULP Project http://iis-projects.ee.ethz.ch/index.php/PULP