Difference between revisions of "Improved Collision Avoidance for Nano-drones"
From iis-projects
Line 39: | Line 39: | ||
https://ieeexplore.ieee.org/abstract/document/8715489 | https://ieeexplore.ieee.org/abstract/document/8715489 | ||
* [2] V. Niculescu et al., "Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs." in IEEE Journal on Emerging and Selected Topics in Circuits and Systems 11, no. 4: 548-562, 2021. https://ieeexplore.ieee.org/abstract/document/9606685 | * [2] V. Niculescu et al., "Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs." in IEEE Journal on Emerging and Selected Topics in Circuits and Systems 11, no. 4: 548-562, 2021. https://ieeexplore.ieee.org/abstract/document/9606685 | ||
− | * [3] https://www.st.com/en/imaging-and-photonics-solutions/vl53l0x.html | + | * [3] Time-of-Flight ranging sensor: https://www.st.com/en/imaging-and-photonics-solutions/vl53l0x.html |
* [4] Bitcraze Crazyflie 2.1 https://www.bitcraze.io/products/crazyflie-2-1/ | * [4] Bitcraze Crazyflie 2.1 https://www.bitcraze.io/products/crazyflie-2-1/ | ||
* [5] PULP Project http://iis-projects.ee.ethz.ch/index.php/PULP | * [5] PULP Project http://iis-projects.ee.ethz.ch/index.php/PULP |
Revision as of 19:45, 15 February 2022
Contents
Description
PULP-Dronet is the flagship of deep learning-based autonomous navigation engines for nano-drones [1,2]. This shallow convolutional neural network (CNN) runs aboard a 27-grams drone being fed with images and outputting a steering angle (regression problem) and a probability of collision (classification problem). The steering angle is then used to keep the center of the lane/street/corridor while the probability of collision enables obstacle avoidance capabilities.
In this project, we want to improve the reliability and accuracy of the obstacle avoidance task by combining the current probability of collision (vision-based) with a pointwise distance measurement enabled by a front-looking time-of-flight (ToF) distance sensor -- STM VL53L0x [3]. The visual cues processed by the deep learning algorithm and precise single-beam distance measurements combine complementary properties which ultimately will lead to a superior collision avoidance capability.
More in the details, the nano-drone platform is the Bitcraze Crazyflie 2.1 [4] extended by a powerful multicore System-on-Chip (SoC), the parallel ultra-low power (PULP) GAP8 [5] aboard the AI-deck companion board [6] and a second companion board called Multiranger-deck [7] which introduces a front-looking ToF sensor. With this setup, the project aims at combining the probability of collision output from the CNN with the ToF measurements so to improve the obstacle avoidance capability by “weighting” both information.
Status: Available
- Available as Master/Semester Thesis (milestones and project's goals will be adjusted accordingly)
- Tutor: Dr. Daniele Palossi dpalossi@iis.ee.ethz.ch
- Supervisor: Prof. Dr. Luca Benini
Prerequisites
- Familiarity with C programming.
- Knowledge of basic python programming and deep learning frameworks (e.g., PyTorch)
- Basic knowledge of parallel programming and Host/Accelerator paradigm, or willing to learn
- Basic knowledge/experience in embedded programming favorable
Character
- 15% Literature and theory study, familarization with existing code
- 30% Python programming, convolutional neural network training
- 30% C embedded programming (STM MCU and PULP programming/optimization)
- 15% In-field test and verification
- 10% Report writing
Meetings
Weekly meetings will be held between the student and the assistants. The exact time and location of these meetings will be determined within the first week of the project. These meetings will be used to evaluate the status and progress of the project. Besides these regular meetings, additional meetings can be organized to address urgent issues as well.
Literature
- [1] D. Palossi et al., "A 64-mW DNN-based visual navigation engine for autonomous nano-drones," in IEEE Internet of Things Journal, 6(5), 8357-8371, 2019.
https://ieeexplore.ieee.org/abstract/document/8715489
- [2] V. Niculescu et al., "Improving Autonomous Nano-Drones Performance via Automated End-to-End Optimization and Deployment of DNNs." in IEEE Journal on Emerging and Selected Topics in Circuits and Systems 11, no. 4: 548-562, 2021. https://ieeexplore.ieee.org/abstract/document/9606685
- [3] Time-of-Flight ranging sensor: https://www.st.com/en/imaging-and-photonics-solutions/vl53l0x.html
- [4] Bitcraze Crazyflie 2.1 https://www.bitcraze.io/products/crazyflie-2-1/
- [5] PULP Project http://iis-projects.ee.ethz.ch/index.php/PULP
- [6] Bitcraze AI-deck 1.1 https://store.bitcraze.io/products/ai-deck-1-1
- [7] Bitcraze Multiranger-deck https://www.bitcraze.io/products/multi-ranger-deck/