Personal tools

Improved Collision Avoidance for Nano-drones

From iis-projects

Revision as of 18:44, 15 February 2022 by Dpalossi (talk | contribs)
Jump to: navigation, search
Overview of the cyber-physical system.

Description

PULP-Dronet is the flagship of deep learning-based autonomous navigation engines for nano-drones [1,2]. This shallow convolutional neural network (CNN) runs aboard a 27-grams drone being fed with images and outputting a steering angle (regression problem) and a probability of collision (classification problem). The steering angle is then used to keep the center of the lane/street/corridor while the probability of collision enables obstacle avoidance capabilities.

In this project, we want to improve the reliability and accuracy of the obstacle avoidance task by combining the current probability of collision (vision-based) with a pointwise distance measurement enabled by a front-looking time-of-flight (ToF) distance sensor -- STM VL53L0x [3]. The visual cues processed by the deep learning algorithm and precise single-beam distance measurements combine complementary properties which ultimately will lead to a superior collision avoidance capability.

More in the details, the nano-drone platform is the Bitcraze Crazyflie 2.1 [4] extended by a powerful multicore System-on-Chip (SoC), the parallel ultra-low power (PULP) GAP8 [5] aboard the AI-deck companion board [6] and a second companion board called Multiranger-deck [7] which introduces a front-looking ToF sensor. With this setup, the project aims at combining the probability of collision output from the CNN with the ToF measurements so to improve the obstacle avoidance capability by “weighting” both information.


Status: Available

Available as Master/Semester Thesis (milestones and project's goals will be adjusted accordingly)
Tutor: Dr. Daniele Palossi dpalossi@iis.ee.ethz.ch
Supervisor: Prof. Dr. Luca Benini

Prerequisites

  • Familiarity with C programming.
  • Knowledge of basic python programming and deep learning frameworks (e.g., PyTorch)
  • Basic knowledge of parallel programming and Host/Accelerator paradigm, or willing to learn
  • Basic knowledge/experience in embedded programming favorable

Character

15% Literature and theory study, familarization with existing code
30% Python programming, convolutional neural network training
30% C embedded programming (STM MCU and PULP programming/optimization)
15% In-field test and verification
10% Report writing

Meetings

Weekly meetings will be held between the student and the assistants. The exact time and location of these meetings will be determined within the first week of the project. These meetings will be used to evaluate the status and progress of the project. Besides these regular meetings, additional meetings can be organized to address urgent issues as well.

Literature

  • [1] D. Palossi et al., "A 64-mW DNN-based visual navigation engine for autonomous nano-drones," in IEEE Internet of Things Journal, 6(5), 8357-8371, 2019.

https://ieeexplore.ieee.org/abstract/document/8715489


Practical Details

↑ top