Personal tools

Difference between revisions of "Improved State Estimation on PULP-based Nano-UAVs"

From iis-projects

Jump to: navigation, search
(Initial version of Project description)
 
(Added detailed task description)
Line 45: Line 45:
  
 
===Meetings & Presentations===
 
===Meetings & Presentations===
The student(s) and advisor(s) agree on weekly meetings to discuss all relevant decisions and decide on how to proceed. Of course, additional meetings can be organized to address urgent issues. <!--
+
Weekly meetings will be held between the student and the assistants.
Around the middle of the project there is a design review, where senior members of the lab review your work (bring all the relevant information, such as prelim. specifications, block diagrams, synthesis reports, testing strategy, ...) to make sure everything is on track and decide whether further support is necessary. They also make the definite decision on whether the chip is actually manufactured (no reason to worry, if the project is on track) and whether more chip area, a different package, ... is provided. For more details confer to [http://eda.ee.ethz.ch/index.php/Design_review].  
+
The exact time and location of these meetings will be determined within the first week of the project in order to fit the students and the assistants schedule.
At the end of the project, you have to present/defend your work during a 15 min. presentation and 5 min. of discussion as part of the IIS colloquium (as required for any semester or master thesis at D-ITET).
+
These meetings will be used to evaluate the status and progress of the project.
 +
Beside these regular meetings, additional meetings can be organized to address urgent issues as well.
 
===Project Goals===
 
===Project Goals===
 
The main goal of this semester thesis is to accelerate the computation of the eKF on nano-drones at the example of a CrazyFlie equipped with a PULP-shield.
 
The main goal of this semester thesis is to accelerate the computation of the eKF on nano-drones at the example of a CrazyFlie equipped with a PULP-shield.
The bonus goal is to improve the eKF, as we have a higher computational capability that enables the use of more sensors. The target application would be to allow UWB error correction, using Gaussian processes based models\cite{UWB}.
+
The bonus goal is to improve the eKF, as we have a higher computational capability that enables the use of more sensors. The target application would be to allow UWB error correction, using Gaussian processes based models[7].
  
The approach we propose for this project is to extend a CrazyFlie with the computational capabilities offered by a PULP-based chip. Contrary to the previous work with the PULP-shield\cite{PULP-DroNet_1}\cite{PULP-DroNet_2}, we aim to use the academic \textit{Mr.Wolf}\cite{mrwolf} PULP-SoC. This SoC features two FPUs on the 8-core cluster, and this enables accelerating the compute-intensive tasks of the flight controller in an ultra-low-power envelope.
+
The approach we propose for this project is to extend a CrazyFlie with the computational capabilities offered by a PULP-based chip. Contrary to the previous work with the PULP-shield[2][3], we aim to use the academic ''Mr.Wolf''[5] PULP-SoC. This SoC features two FPUs on the 8-core cluster, and this enables accelerating the compute-intensive tasks of the flight controller in an ultra-low-power envelope.
  
 
On this platform, we can implement the matrix multiplications of the eKF to be executed in parallel, speeding up the execution.  
 
On this platform, we can implement the matrix multiplications of the eKF to be executed in parallel, speeding up the execution.  
Line 58: Line 59:
 
The main tasks of this project are:
 
The main tasks of this project are:
  
* T1: Designing and implementing a SPI bi-directional data transfer between Mr.Wolf and STM32, for exchanging sensor data and results
+
* '''T1:''' Designing and implementing a SPI bi-directional data transfer between Mr.Wolf and STM32, for exchanging sensor data and results
* T2: Porting the eKF used on the Crazyflie to Mr.Wolf (sequential execution on one core)
+
* '''T2:''' Porting the eKF used on the Crazyflie to Mr.Wolf (sequential execution on one core)
* T3: Verifying the eKF computation on PULP  
+
* '''T3:''' Verifying the eKF computation on PULP  
* T4: Parallelization of the ekF
+
* '''T4:''' Parallelization of the ekF
* T5: (optional) Fusing UWB range measurements into the eKF
+
* '''T5:''' (optional) Fusing UWB range measurements into the eKF
  
 
===Deliverables===
 
===Deliverables===
* description of the most promising architectures, and argumentation on the decision taken (as part of the report)
+
* Communication protocol between PULP/STM32
* synthesizable, verified VHDL code
+
* Sequential eKF implementation on PULP
* generated test vector files
+
* Parallel eKF implementation on PULP
* synthesis scripts & relevant software models developed for verification
+
* (optional) Parallel eKF with UWB on PULP
 
* synthesis results and final chip layout (GDS II data), bonding diagram
 
* synthesis results and final chip layout (GDS II data), bonding diagram
* datasheet (part of report)
+
* Results: Power and performance measurements, accuracy evaluation
* presentation slides
+
* Presentation and report including software documentation
* project report (in digital form; a hard copy also welcome, but not necessary)
+
===Timeline===
===Timeline==
 
 
To give some idea on how the time can be split up, we provide some possible partitioning:  
 
To give some idea on how the time can be split up, we provide some possible partitioning:  
* Literature survey, building a basic understanding of the problem at hand, catch up on related work
+
* Familiarize with Crazyflie (CF) firmware (1 weeks)
* Development of a working software-based implementation running on the Zynq's ARM core
+
* Implement data transfer between CF and PULP (1 week)
* Piece-by-piece off-loading of relevant tasks to the programmable logic
+
* Porting/Debugging/Verify the eKF to PULP (4 weeks)
* Implementation of data interfaces (software or hardware)
+
* Parallelize PULP implementation (3 weeks)
* Report and presentation  
+
* (optional) Fusing UWB range measurements into the eKF (2 weeks)
-->
+
* Preparing thesis report and final presentation (2 weeks)
<!-- 13.5 weeks total here -->
+
<!-- 13 weeks total here -->
  
 
===Literature===
 
===Literature===

Revision as of 11:02, 24 June 2020

Figure 1: A-B) The PULP-Shield PCB developed at IIS by our student Hanna Müller. C) Our nano-drone prototype based on the CrazyFlie 2.0 coupled with the PULP-Shield.

Description

Autonomous unmanned aerial vehicles (UAVs) are increasingly getting smaller and smarter. Their tiny form factor can be a game-changer in many practical applications such as the aerial inspection or assistance in natural disaster areas, to name a few.

In this context, the parallel ultra-low-power (PULP) platform[4] developed here at IIS, is the fundamental computational unit to bring state-of-the-art complex algorithms for autonomous navigation into the nano-scale class of vehicles. There are several embodiment of the PULP paradigm, one of them a system-on-chip (SoC) called Mr.Wolf[5]. This SoC features 9 cores, divided into a fabric controller and a 8 core cluster with access to 2 floating point units (FPUs).

A-B) Figure 2: Overview of control tasks execution flow.

To control any robot there have to be three components implemented:

  • State estimation current state A
  • Intelligence next state B
  • Actuation get from A to B

Previous work with the PULP-Shield, a tiny pluggable PCB, featuring a PULP-based SoC, focused on the intelligence, using a CNN to determine where to fly aboard a commercial Crazyflie 2.0 nano-drone[1][2][3]. In this thesis, instead, we want to focus on the first component - the state estimation. To estimate where the drone is, we need to fuse the results of the on-board sensors to compute the position and attitude the drone is in. We want to use an extended Kalman filter (eKF), a non-linear estimator, for this task. This project will base the implementation of the eKF on the already used one on the Crazyflie[6].

The goal of this thesis is to port, optimize, and improve state estimation with a PULP SoC. Starting from the eKF used on the Crazyflie[6], we want to offload this compute-intensive task from the STM32 to a PULP SoC. This will allow us to incorporate even more sensor measurements into the eKF. The use case we want to implement is to improve the state estimation by fusing also ultra-wideband (UWB) range measurements[7]. Figure \ref{fig:SW_overview} shows the resulting execution flow.

The resulting eKF can also be adopted on different PULP-based hardware designs.

Status: Available

Available as Master Thesis for one student
Available as Semester Thesis for one or two students
Supervisor: Daniele Palossi
Professor: Luca Benini

Prerequisites

  • Familiarity with embedded system programming in C.
  • Basic knowledge of parallel programming and Host/Accelerator paradigm.
  • Basic knowledge of FreeRTOS [2] and STM32F4 MCU family [3] is favorable.

Character

20% Literature and theory study
10% C programming on x86
30% C embedded programming (PULP programming/optimization)
20% Test and verification on the real prototype
20% Experimental evaluation and report writin

Detailed Task Description

Meetings & Presentations

Weekly meetings will be held between the student and the assistants. The exact time and location of these meetings will be determined within the first week of the project in order to fit the students and the assistants schedule. These meetings will be used to evaluate the status and progress of the project. Beside these regular meetings, additional meetings can be organized to address urgent issues as well.

Project Goals

The main goal of this semester thesis is to accelerate the computation of the eKF on nano-drones at the example of a CrazyFlie equipped with a PULP-shield. The bonus goal is to improve the eKF, as we have a higher computational capability that enables the use of more sensors. The target application would be to allow UWB error correction, using Gaussian processes based models[7].

The approach we propose for this project is to extend a CrazyFlie with the computational capabilities offered by a PULP-based chip. Contrary to the previous work with the PULP-shield[2][3], we aim to use the academic Mr.Wolf[5] PULP-SoC. This SoC features two FPUs on the 8-core cluster, and this enables accelerating the compute-intensive tasks of the flight controller in an ultra-low-power envelope.

On this platform, we can implement the matrix multiplications of the eKF to be executed in parallel, speeding up the execution.

The main tasks of this project are:

  • T1: Designing and implementing a SPI bi-directional data transfer between Mr.Wolf and STM32, for exchanging sensor data and results
  • T2: Porting the eKF used on the Crazyflie to Mr.Wolf (sequential execution on one core)
  • T3: Verifying the eKF computation on PULP
  • T4: Parallelization of the ekF
  • T5: (optional) Fusing UWB range measurements into the eKF

Deliverables

  • Communication protocol between PULP/STM32
  • Sequential eKF implementation on PULP
  • Parallel eKF implementation on PULP
  • (optional) Parallel eKF with UWB on PULP
  • synthesis results and final chip layout (GDS II data), bonding diagram
  • Results: Power and performance measurements, accuracy evaluation
  • Presentation and report including software documentation

Timeline

To give some idea on how the time can be split up, we provide some possible partitioning:

  • Familiarize with Crazyflie (CF) firmware (1 weeks)
  • Implement data transfer between CF and PULP (1 week)
  • Porting/Debugging/Verify the eKF to PULP (4 weeks)
  • Parallelize PULP implementation (3 weeks)
  • (optional) Fusing UWB range measurements into the eKF (2 weeks)
  • Preparing thesis report and final presentation (2 weeks)

Literature

  • [1] Crazyflie2.0 http://www.bitcraze.io/crazyflie-2/
  • [2] D. Palossi et al., "A 64mW DNN-based Visual Navigation Engine for Au-tonomous Nano-Drones", in IEEE Internet of Things Journal, 2019. doi:10.1109/JIOT.2019.2917066
  • [3] D. Palossi, F. Conti and L. Benini, "An Open Source and Open Hardware DeepLearning-Powered Visual Navigation Engine for Autonomous Nano-UAVs", 2019 15thInternational Conference on Distributed Computing in Sensor Systems (DCOSS),Santorini Island, Greece, 2019, pp. 604-611. doi: 10.1109/DCOSS.2019.00111
  • [4] PULP Project http://iis-projects.ee.ethz.ch/index.php/PULP
  • [5] A. Pullini, D. Rossi, I. Loi, A. Di Mauro and L. Benini, "Mr. Wolf: A 1 GFLOP/sEnergy-Proportional Parallel Ultra Low Power SoC for IOT Edge Processing", ES-SCIRC 2018 - IEEE 44th European Solid State Circuits Conference (ESSCIRC),Dresden, 2018, pp. 274-277. doi: 10.1109/ESSCIRC.2018.8494247
  • [6] M. Mueller, M. Hamer, R. D’Andrea,"Fusing ultra-wideband range measurementswith accelerometers and rate gyroscopes for quadrocopter state estimation" 2015IEEE International Conference on Robotics and Automation (ICRA)
  • [7] A. Ledergerber, R. D’Andrea, "Ultra-wideband range measurement model with Gaus-sian processes", 2017 IEEE Conference on Control Technology and Applications(CCTA),2017, pp. 1929-1934.


Practical Details

↑ top