Personal tools

Difference between revisions of "Energy Efficient Autonomous UAVs"

From iis-projects

Jump to: navigation, search
(Contact Information)
(Contact Information)
Line 91: Line 91:
  
 
=====Hanna Mueller=====
 
=====Hanna Mueller=====
[[File:tommaso.jpg|thumb|right|150px|[[:User:Tommaso | Dr. Tommaso Polonelli]]]]
+
[[File:tommaso.jpeg|thumb|right|150px|[[:User:Tommaso | Dr. Tommaso Polonelli]]]]
  
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch tommaso.polonelli@pbl.ee.ethz.ch]
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch tommaso.polonelli@pbl.ee.ethz.ch]

Revision as of 14:02, 13 June 2022

A-B) The PULP-Shield PCB developed at IIS by our student Hanna Müller. C) Our nano-drone prototype based on the CrazyFlie 2.0 coupled with the PULP-Shield.


Introduction and Platforms

The interest in autonomous vehicles is growing constantly, with lots of practical applications appearing on the marketplace and many more being actively studied in academia, industry, and military research departments. Two of the most representative examples of such technology are unmanned aerial vehicles (UAVs) and autonomous driving systems. Autonomous systems are emerging in many fields in order to assist and help humans in a plethora of applications, like environment surveillance, inspection, monitoring, etc. Fully autonomous driving is still out of reach, but virtually every major OEM (BMW, Volvo, Tesla Motors, GM) has a clear roadmap towards achieving this goal and is already commercializing advanced driver assistance systems (ADAS). Autonomous vehicles represent a unique opportunity to effectively handle critical and expensive activities, compared to the traditional use of human operators only.

When it comes to performing autonomous navigation, widely adopted tasks, such as obstacle avoidance, path planning, target/pedestrian detection, simultaneous localization, and mapping, etc., are very complex and computationally intensive. Given the high computational requirements that such workloads expose, current state-of-the-art solutions need quite powerful computational units. The typical class of devices used for such sophisticated algorithms are multi-cores high-end CPUs and embedded heterogeneous systems featuring powerful CPUs coupled with many-core accelerators (e.g., GPUs). In both classes, the order of magnitude for the power consumption is in the range of few Watts to tens of Watts.

Such computational requirements, and power budget, limit the applicability of such techniques to vehicles that expose enough power for the computation. If we consider for instance rotorcraft or flapping-wings UAVs, the available power budget for computation is between 5-20% of the overall power. Thus, reducing the power consumption of the control system in UAVs will become increasingly important as the size of the vehicle is scaled down. Indeed, reducing the UAV size quickly leads to an order-of-magnitude reduction of the power spent on the propellers. Besides, the computational load required to implement the UAV cognitive skills does not vary with the vehicle size, and will thus constitute an increasingly larger fraction of the total system power consumption.

a. Pico-size Quadrotor (Fünfliber drone)

The Fünfliber-drone

The Fünfliber drone contains the Mr.Wolf SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: one, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. In contrast to the commercial GWT GAP8, Mr.Wolf features a Floating-Point Unit on the cluster. Furthermore, the drone features 6 Time of Flight sensors, a Himax camera, an optical flow sensor, a barometer, an IMU, an infrared camera and a radio chip.


.

b. Nano-size Quadrotor

The CrazyFlie 2.1

Our main cyber-physical platform is the 27g nano-size Bitcraze CrazyFlie 2.1 [1],. It has been used in many projects due to its dimension, versatility, and its open-source and open-hardware nature. Nowadays is well known how UAVs with high-level autonomous navigation capabilities are a hot topic both in industry and academia due to their numerous applications. However, autonomous navigation algorithms are demanding from a computational standpoint, and it is very challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter) because of the limited capabilities of their MCU-based controllers that are also in charge of running the control and estimation algorithms for flying the drone. In order to extend the computational capabilities of the drone, we equip it with the plug-in PC called the AI-Deck that enables complex artificial intelligence-based workloads to run onboard, with the possibility to achieve fully autonomous navigation capabilities.

The AI-deck contains the GWT GAP8 SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: the first, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. Furthermore, the AI-deck also features an ultra-low-power onboard camera and a WiFi module.

Our previous work presents a lightweight hardware-software solution based on a CNN that brings autonomous navigation on the Crazyflie+Ai-deck using only the onboard resources. Furthermore, we evaluated how the Parallel Ultra-Low-Power Platform [2] can enable running sophisticated machine learning algorithms for achieving autonomous navigation or localization capabilities.

Demo Video


.

Topics

a. Vision-based Autonomous Navigation

The CrazyFlie equipped with the AI-deck performing autonomous navigation

Vision-based perception algorithms traditionally employ simultaneous localization-and-mapping (SLAM), which is a technique that builds a 3D local map of the environment, which is used to plan the trajectory accordingly. Nevertheless, the main drawback of the approaches based on SLAM is that they are computationally demanding. Furthermore, while SLAM can be used for perception, it does not solve the challenging issue of inferring the control commands from the 3D map. More recent solutions, such as end-to-end convolutional neural networks (CNNs), mitigate this issue by directly estimating the optimal control commands, using camera images as input. Due to the recent advancement of low-power parallel hardware platforms such as PULP, convolutional neural networks (CNNs) can run very efficiently, even aboard a small nano-drone. Therefore, the GAP8 chip found on the AI-deck allows running complex NN models significantly faster than the classical single-core MCUs (such as the Cortex M4 found in Crazyflie).


.

b. Lidar/Radar-based Autonomous Navigation

As an alternative to camera-based navigation methods, solutions based on distance estimation gained more and more popularity for accomplishing obstacle avoidance. This is mainly due to the technological advancement of optical time-of-flight (ToF) sensors. This class of sensors estimates the distance by emitting a light beam towards a target and measuring when the partially reflected beam arrives. Based on this, the sensor calculates the roundtrip time and determines the distance. These sensors were widely used in applications such as proximity sensing or presence detection. Moreover, they are also found in drones, where they enable the drone to estimate the height with respect to the ground for indoor applications. More recent solutions that employ this technology, include several such sensors grouped in a matrix, aiming to perform multi-object detection. Therefore, the output of such a sensor will be a low-resolution "image" that contains depth information. Compared to cameras, this approach has the advantage of working reliably in poor-light conditions.

C. Indoor Localization with UWB

Localization.jpg

Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters. Estimation algorithms such as Kalman filters or multilateration (based on cost function optimization) are used to determine the position of the drone, given the anchor locations and the UWB measurements. b) presents the dual scenario, where the drone is used as a moving anchor and acquires range measurements in various spots to determine the location of a static, unknown position UWB node.

However, there are still drawbacks associated with UWB: for instance, in some situations, physical factors (i.e., obstacles in the line-of-sight) or such antenna delays can significantly decrease the UWB ranging accuracy. Furthermore, since the ToA ranging is sequential, increasing the number of drones will result in fewer UWB measurements per drone (since the maximum ranging frequency is divided by the number of drones). Sophisticated learning algorithms can mitigate these issues, by modeling the UWB ranging errors and using the model to compensate for the errors. Moreover, learning algorithms can identify patterns in the UWB channel impulse response (CIR) and therefore enable features such as concurrent ranging and angle-of-arrival estimation.

The goal is to exploit the capabilities of the parallel chip (i.e., GAP8) found on the AI-deck to run algorithms that improve the localization accuracy in applications with drones and swarms.

Contact Information

Hanna Mueller


Vlad Niculescu


Dr. Daniele Palossi
Hanna Mueller


Projects Overview

Available Projects


Publications

The group effort and the great contribution from the students of last few years has resulted in the following list of publications:

  • 2022 - V. Niculescu et Al., "Fly, wake-up, find: UAV-based energy-efficient localization for distributed sensor nodes", Sustainable Computing: Informatics and Systems, - Link
  • 2021 - V. Niculescu et Al., "Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, - Link
  • 2021 - H. Mueller et Al., "Fünfiiber-Drone: A Modular Open-Platform 18-grams Autonomous Nano-Drone", 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), - Link
  • 2021 - V. Niculescu et Al., "Automated Tuning of End-to-end Neural Flight Controllers for Autonomous Nano-drones", 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), - Link
  • 2020 - V. Niculescu et Al., "An Energy-efficient Localization System for Imprecisely Positioned Sensor Nodes with Flying UAVs", 2020 IEEE 18th International Conference on Industrial Informatics (INDIN), - Link
  • 2017 - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", 7th IEEE International Workshop on Advances in Sensors and Interfaces, June 15-16, Vieste, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", 20th International Workshop on Software and Compilers for Embedded Systems (SCOPES), June 12-13, Sankt Goar, Germany, 2017 - On-line
  • 2017 - D. Palossi et Al., "Self-sustainability in Nano Unmanned Aerial Vehicles: A Blimp Case Study", Computing Frontiers (CF), May 15-17, Siena, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "Ultra Low-Power Visual Odometry for Nano-Scale Unmanned Aerial Vehicles", Design, Automation and Test in Europe (DATE), March 27-31, Lausanne, Switzerland, 2017 - On-line
  • 2016 - D. Palossi et Al., "Exploring Single Source Shortest Path Parallelization on Shared Memory Accelerator", 19th International Workshop on Software and Compilers for Embedded Systems (SCOPES), May 23-25, Sankt Goar, Germany, 2016 - On-line
  • 2016 - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", 2nd Workshop on Design of Low Power Embedded Systems (LP-EMS), May 16-18, Como, Italy, 2016 - On-line

External Links


↑ top