Event-based navigation on autonomous nano-drones
From iis-projects
Contents
Description
Autonomous nano-drones, i.e., as big as the palm of your hand, are increasingly getting massive attention from Academia and industry. Their tiny form factor can be a game-changer in many practical applications such as aerial inspection or assistance in natural disaster areas, to name a few. This project follows the trail of the PULP-Dronet research project [1,2]: a deep learning-based autonomous navigation engine for nano-drones. As the evolution of this research domain, we envision a new generation of autonomous nano-drones that combines deep learning-based (DL) brains with novel event-based dynamic vision sensors (DVS). In this scenario, the DVS sensor will cover the role of a primary source of high-throughput visual information to be processed directly onboard a resource-constrained nano-drone. Therefore, the candidate will have the opportunity to design a new DL-based algorithm to run on a novel ultra-low-power processor, such as the PULP Kraken [3,4] System-on-Chip (SoC). At the same time, the candidate will also work on the hardware design of a companion board to host this SoC aboard a 10-cm nano-drone [5].
Status: Available
- Available as Master/Semester Thesis (milestones and project's goals will be adjusted accordingly)
- Tutor: Dr. Alfio Di Mauro adimauro@iis.ee.ethz.ch Dr. Daniele Palossi dpalossi@iis.ee.ethz.ch
- Supervisor: Prof. Dr. Luca Benini
Prerequisites
- Familiarity with C programming.
- PCB design background.
- Knowledge of basic python programming and deep learning frameworks (e.g., PyTorch).
- Knowledge/experience in embedded programming and/or parallel programming is favorable.
Character
- 15% Literature and theory study, familarization with existing code
- 10% Python programming, CNN architecture design
- 25% C embedded programming (STM MCU and PULP programming/optimization)
- 25% PCB design
- 15% In-field test and verification
- 10% Report writing
Meetings
Weekly meetings will be held between the student and the assistants. The exact time and location of these meetings will be determined within the first week of the project. These meetings will be used to evaluate the status and progress of the project. Besides these regular meetings, additional meetings can be organized to address urgent issues as well.
Literature
- [1] D. Palossi et al., "A 64-mW DNN-based visual navigation engine for autonomous nano-drones," in IEEE IoT Journal, 2019. https://ieeexplore.ieee.org/abstract/document/8715489
- [2] V. Niculescu et al., "Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs." in IEEE JETCAS, 2021. https://ieeexplore.ieee.org/abstract/document/9606685
- [3] SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions. https://arxiv.org/abs/2204.10687
- [4] CUTIE: Beyond PetaOp/s/W Ternary DNN Inference Acceleration with Better-than-Binary Energy Efficiency. https://arxiv.org/abs/2011.01713
- [5] Bitcraze Crazyflie 2.1 https://www.bitcraze.io/products/crazyflie-2-1/