Personal tools

Difference between revisions of "Energy Efficient Autonomous UAVs"

From iis-projects

Jump to: navigation, search
(Available Projects)
 
(20 intermediate revisions by 2 users not shown)
Line 33: Line 33:
 
[https://www.youtube.com/watch?v=VPeSnjoGDEc Demo Video]
 
[https://www.youtube.com/watch?v=VPeSnjoGDEc Demo Video]
  
===c. Standard-size Quadrotor===
 
 
[[File:std_size_drone.png|thumb|right|200px| The Standard-size Platform]]
 
 
The Standard-Size Unmanned Aerial Vehicle platform is a complete system that provides a development testbed for implementing and profiling advanced algorithms for autonomous navigation. Its primary purpose is to provide an unconstrained environment (computational power, sensor accuracy, and battery duration) to test new algorithm paradigms later to be applied, with further optimizations, in other drone architectures such as the nano-sized drone platform. Additionally, the platform can also serve as the central platform for other autonomous navigation scenarios where its unique features are required. At the center of the architecture, the MATEK F722-SE (ARM Cortex-M7) Flight Controller (FC) together with iNav, an open-source firmware platform for autonomous navigation, account for the flight’s attitude control, telemetry forwarding, and communication with external devices, such as base stations or onboard computational modules. The architecture also integrates a high-end embedded device (NVIDIA Jetson TX2) that provides computational power and memory in excess to satisfy the real-time constraints of the processing of state-of-the-art(SoA) algorithms for autonomous navigation.
 
  
  
Line 57: Line 52:
  
 
===C. Indoor Localization with UWB===
 
===C. Indoor Localization with UWB===
[[File:localization.jpg|thumb|right|320px| test]]
+
[[File:localization.jpg|thumb|right|320px| ]]
  
 
Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters.
 
Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters.
Line 72: Line 67:
  
 
=====Hanna Mueller=====
 
=====Hanna Mueller=====
 +
[[File:hanna.jpg|thumb|right|150px|[[:User:Hanna | Hanna Mueller]]]]
  
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch hanmuell@iis.ee.ethz.ch]
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch hanmuell@iis.ee.ethz.ch]
Line 78: Line 74:
 
* '''office''': ETZ J76.2
 
* '''office''': ETZ J76.2
  
[[File:vlad_profile.jpg|thumb|right|210px|[[:User:Vladn | Vlad Niculescu]]]]
+
 
 +
[[File:bio_vlad2.JPG|thumb|right|150px|[[:User:Vladn | Vlad Niculescu]]]]
 
=====Vlad Niculescu=====
 
=====Vlad Niculescu=====
  
Line 86: Line 83:
  
  
[[File:DP-UAV.jpeg|thumb|right|210px|[[:User:Dpalossi | Daniele Palossi]]]]
+
[[File:DP21.png|thumb|right|150px|[[:User:Dpalossi | Dr. Daniele Palossi]]]]
=====Daniele Palossi=====
+
=====Dr. Daniele Palossi=====
  
 
* '''e-mail''': [mailto:dpalossi@iis.ee.ethz.ch dpalossi@iis.ee.ethz.ch]
 
* '''e-mail''': [mailto:dpalossi@iis.ee.ethz.ch dpalossi@iis.ee.ethz.ch]
* '''phone''': +41 44 633 88 43
 
 
* '''address''': Gloriastrasse 35, 8092 Zürich
 
* '''address''': Gloriastrasse 35, 8092 Zürich
* '''office''': ETZ J76.2
+
* '''office''': ETZ J65
 +
 
  
 
==Projects Overview==
 
==Projects Overview==
Line 98: Line 95:
 
<DynamicPageList>
 
<DynamicPageList>
 
category = Available
 
category = Available
 +
category = Digital
 
category = UAV
 
category = UAV
category = Digital
 
 
suppresserrors=true
 
suppresserrors=true
 
</DynamicPageList>
 
</DynamicPageList>
Line 106: Line 103:
 
The group effort and the great contribution from the students of last few years has resulted in the following list of publications:
 
The group effort and the great contribution from the students of last few years has resulted in the following list of publications:
  
* '''2018''' - D. Palossi et Al., "Extending the Lifetime of Nano-Blimps via Dynamic Motor Control", ''Springer Journal of Signal Processing Systems (Springer JSPS)'', 2018 - [https://link.springer.com/article/10.1007/s11265-018-1343-1 On-line]
+
* '''2022''' - V. Niculescu et Al., "Fly, wake-up, find: UAV-based energy-efficient localization for distributed sensor nodes", ''Sustainable Computing: Informatics and Systems'', - [https://www.sciencedirect.com/science/article/pii/S2210537922000038 Link]
 +
 
 +
* '''2021''' - V. Niculescu et Al., "Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs", ''IEEE Journal on Emerging and Selected Topics in Circuits and Systems'', - [https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9606685 Link]
 +
 
 +
* '''2021''' - H. Mueller et Al., "Fünfiiber-Drone: A Modular Open-Platform 18-grams Autonomous Nano-Drone", ''2021 Design, Automation & Test in Europe Conference & Exhibition (DATE)'', - [https://ieeexplore.ieee.org/abstract/document/9474262 Link]
 +
 
 +
* '''2021''' - V. Niculescu et Al., "Automated Tuning of End-to-end Neural Flight Controllers for Autonomous Nano-drones", ''2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS)'', - [https://ieeexplore.ieee.org/abstract/document/9458550 Link]
 +
 
 +
* '''2020''' - V. Niculescu et Al., "An Energy-efficient Localization System for Imprecisely Positioned Sensor Nodes with Flying UAVs", ''2020 IEEE 18th International Conference on Industrial Informatics (INDIN)'', - [https://ieeexplore.ieee.org/abstract/document/9442075 Link]
  
 
* '''2017''' - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", ''7th IEEE International Workshop on Advances in Sensors and Interfaces'', June 15-16, Vieste, Italy, 2017 - [http://ieeexplore.ieee.org/document/7974242/ On-line]
 
* '''2017''' - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", ''7th IEEE International Workshop on Advances in Sensors and Interfaces'', June 15-16, Vieste, Italy, 2017 - [http://ieeexplore.ieee.org/document/7974242/ On-line]
 
* '''2017''' - B. Forsberg et Al., "GPU-Accelerated Real-Time Path Planning and the Predictable Execution Model", ''International Conference on Computational Science (ICCS)'', June 12-14, Zürich, Switzerland, 2017 - [http://www.sciencedirect.com/science/article/pii/S1877050917308256 On-line]
 
  
 
* '''2017''' - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", ''20th International Workshop on Software and Compilers for Embedded Systems (SCOPES)'', June 12-13, Sankt Goar, Germany, 2017 - [http://dl.acm.org/citation.cfm?id=3079072 On-line]
 
* '''2017''' - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", ''20th International Workshop on Software and Compilers for Embedded Systems (SCOPES)'', June 12-13, Sankt Goar, Germany, 2017 - [http://dl.acm.org/citation.cfm?id=3079072 On-line]
Line 121: Line 124:
  
 
* '''2016''' - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", ''2nd Workshop on Design of Low Power Embedded Systems (LP-EMS)'', May 16-18, Como, Italy, 2016 - [http://dl.acm.org/citation.cfm?id=2911712 On-line]
 
* '''2016''' - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", ''2nd Workshop on Design of Low Power Embedded Systems (LP-EMS)'', May 16-18, Como, Italy, 2016 - [http://dl.acm.org/citation.cfm?id=2911712 On-line]
 
* '''2016''' - F. Conti et Al., "Enabling the Heterogeneous Accelerator Model on Ultra-Low Power Microcontroller Platforms", ''Design, Automation and Test in Europe (DATE)'', March 14-18, Dresden, Germany, 2016 - [http://ieeexplore.ieee.org/document/7459494/ On-line]
 
 
 
  
 
=External Links=
 
=External Links=

Latest revision as of 16:18, 15 March 2022

A-B) The PULP-Shield PCB developed at IIS by our student Hanna Müller. C) Our nano-drone prototype based on the CrazyFlie 2.0 coupled with the PULP-Shield.


Introduction and Platforms

The interest in autonomous vehicles is growing constantly, with lots of practical applications appearing on the marketplace and many more being actively studied in academia, industry, and military research departments. Two of the most representative examples of such technology are unmanned aerial vehicles (UAVs) and autonomous driving systems. Autonomous systems are emerging in many fields in order to assist and help humans in a plethora of applications, like environment surveillance, inspection, monitoring, etc. Fully autonomous driving is still out of reach, but virtually every major OEM (BMW, Volvo, Tesla Motors, GM) has a clear roadmap towards achieving this goal and is already commercializing advanced driver assistance systems (ADAS). Autonomous vehicles represent a unique opportunity to effectively handle critical and expensive activities, compared to the traditional use of human operators only.

When it comes to performing autonomous navigation, widely adopted tasks, such as obstacle avoidance, path planning, target/pedestrian detection, simultaneous localization, and mapping, etc., are very complex and computationally intensive. Given the high computational requirements that such workloads expose, current state-of-the-art solutions need quite powerful computational units. The typical class of devices used for such sophisticated algorithms are multi-cores high-end CPUs and embedded heterogeneous systems featuring powerful CPUs coupled with many-core accelerators (e.g., GPUs). In both classes, the order of magnitude for the power consumption is in the range of few Watts to tens of Watts.

Such computational requirements, and power budget, limit the applicability of such techniques to vehicles that expose enough power for the computation. If we consider for instance rotorcraft or flapping-wings UAVs, the available power budget for computation is between 5-20% of the overall power. Thus, reducing the power consumption of the control system in UAVs will become increasingly important as the size of the vehicle is scaled down. Indeed, reducing the UAV size quickly leads to an order-of-magnitude reduction of the power spent on the propellers. Besides, the computational load required to implement the UAV cognitive skills does not vary with the vehicle size, and will thus constitute an increasingly larger fraction of the total system power consumption.

a. Pico-size Quadrotor (Fünfliber drone)

The Fünfliber-drone

The Fünfliber drone contains the Mr.Wolf SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: one, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. In contrast to the commercial GWT GAP8, Mr.Wolf features a Floating-Point Unit on the cluster. Furthermore, the drone features 6 Time of Flight sensors, a Himax camera, an optical flow sensor, a barometer, an IMU, an infrared camera and a radio chip.


.

b. Nano-size Quadrotor

The CrazyFlie 2.1

Our main cyber-physical platform is the 27g nano-size Bitcraze CrazyFlie 2.1 [1],. It has been used in many projects due to its dimension, versatility, and its open-source and open-hardware nature. Nowadays is well known how UAVs with high-level autonomous navigation capabilities are a hot topic both in industry and academia due to their numerous applications. However, autonomous navigation algorithms are demanding from a computational standpoint, and it is very challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter) because of the limited capabilities of their MCU-based controllers that are also in charge of running the control and estimation algorithms for flying the drone. In order to extend the computational capabilities of the drone, we equip it with the plug-in PC called the AI-Deck that enables complex artificial intelligence-based workloads to run onboard, with the possibility to achieve fully autonomous navigation capabilities.

The AI-deck contains the GWT GAP8 SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: the first, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. Furthermore, the AI-deck also features an ultra-low-power onboard camera and a WiFi module.

Our previous work presents a lightweight hardware-software solution based on a CNN that brings autonomous navigation on the Crazyflie+Ai-deck using only the onboard resources. Furthermore, we evaluated how the Parallel Ultra-Low-Power Platform [2] can enable running sophisticated machine learning algorithms for achieving autonomous navigation or localization capabilities.

Demo Video


.

Topics

a. Vision-based Autonomous Navigation

The CrazyFlie equipped with the AI-deck performing autonomous navigation

Vision-based perception algorithms traditionally employ simultaneous localization-and-mapping (SLAM), which is a technique that builds a 3D local map of the environment, which is used to plan the trajectory accordingly. Nevertheless, the main drawback of the approaches based on SLAM is that they are computationally demanding. Furthermore, while SLAM can be used for perception, it does not solve the challenging issue of inferring the control commands from the 3D map. More recent solutions, such as end-to-end convolutional neural networks (CNNs), mitigate this issue by directly estimating the optimal control commands, using camera images as input. Due to the recent advancement of low-power parallel hardware platforms such as PULP, convolutional neural networks (CNNs) can run very efficiently, even aboard a small nano-drone. Therefore, the GAP8 chip found on the AI-deck allows running complex NN models significantly faster than the classical single-core MCUs (such as the Cortex M4 found in Crazyflie).


.

b. Lidar/Radar-based Autonomous Navigation

As an alternative to camera-based navigation methods, solutions based on distance estimation gained more and more popularity for accomplishing obstacle avoidance. This is mainly due to the technological advancement of optical time-of-flight (ToF) sensors. This class of sensors estimates the distance by emitting a light beam towards a target and measuring when the partially reflected beam arrives. Based on this, the sensor calculates the roundtrip time and determines the distance. These sensors were widely used in applications such as proximity sensing or presence detection. Moreover, they are also found in drones, where they enable the drone to estimate the height with respect to the ground for indoor applications. More recent solutions that employ this technology, include several such sensors grouped in a matrix, aiming to perform multi-object detection. Therefore, the output of such a sensor will be a low-resolution "image" that contains depth information. Compared to cameras, this approach has the advantage of working reliably in poor-light conditions.

C. Indoor Localization with UWB

Localization.jpg

Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters. Estimation algorithms such as Kalman filters or multilateration (based on cost function optimization) are used to determine the position of the drone, given the anchor locations and the UWB measurements. b) presents the dual scenario, where the drone is used as a moving anchor and acquires range measurements in various spots to determine the location of a static, unknown position UWB node.

However, there are still drawbacks associated with UWB: for instance, in some situations, physical factors (i.e., obstacles in the line-of-sight) or such antenna delays can significantly decrease the UWB ranging accuracy. Furthermore, since the ToA ranging is sequential, increasing the number of drones will result in fewer UWB measurements per drone (since the maximum ranging frequency is divided by the number of drones). Sophisticated learning algorithms can mitigate these issues, by modeling the UWB ranging errors and using the model to compensate for the errors. Moreover, learning algorithms can identify patterns in the UWB channel impulse response (CIR) and therefore enable features such as concurrent ranging and angle-of-arrival estimation.

The goal is to exploit the capabilities of the parallel chip (i.e., GAP8) found on the AI-deck to run algorithms that improve the localization accuracy in applications with drones and swarms.

Contact Information

Hanna Mueller


Vlad Niculescu


Dr. Daniele Palossi


Projects Overview

Available Projects


Publications

The group effort and the great contribution from the students of last few years has resulted in the following list of publications:

  • 2022 - V. Niculescu et Al., "Fly, wake-up, find: UAV-based energy-efficient localization for distributed sensor nodes", Sustainable Computing: Informatics and Systems, - Link
  • 2021 - V. Niculescu et Al., "Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, - Link
  • 2021 - H. Mueller et Al., "Fünfiiber-Drone: A Modular Open-Platform 18-grams Autonomous Nano-Drone", 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), - Link
  • 2021 - V. Niculescu et Al., "Automated Tuning of End-to-end Neural Flight Controllers for Autonomous Nano-drones", 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), - Link
  • 2020 - V. Niculescu et Al., "An Energy-efficient Localization System for Imprecisely Positioned Sensor Nodes with Flying UAVs", 2020 IEEE 18th International Conference on Industrial Informatics (INDIN), - Link
  • 2017 - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", 7th IEEE International Workshop on Advances in Sensors and Interfaces, June 15-16, Vieste, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", 20th International Workshop on Software and Compilers for Embedded Systems (SCOPES), June 12-13, Sankt Goar, Germany, 2017 - On-line
  • 2017 - D. Palossi et Al., "Self-sustainability in Nano Unmanned Aerial Vehicles: A Blimp Case Study", Computing Frontiers (CF), May 15-17, Siena, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "Ultra Low-Power Visual Odometry for Nano-Scale Unmanned Aerial Vehicles", Design, Automation and Test in Europe (DATE), March 27-31, Lausanne, Switzerland, 2017 - On-line
  • 2016 - D. Palossi et Al., "Exploring Single Source Shortest Path Parallelization on Shared Memory Accelerator", 19th International Workshop on Software and Compilers for Embedded Systems (SCOPES), May 23-25, Sankt Goar, Germany, 2016 - On-line
  • 2016 - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", 2nd Workshop on Design of Low Power Embedded Systems (LP-EMS), May 16-18, Como, Italy, 2016 - On-line

External Links


↑ top