Personal tools

Difference between revisions of "Energy Efficient Autonomous UAVs"

From iis-projects

Jump to: navigation, search
(→‎Nano-size Quadrotor)
(→‎Contact Information)
 
(119 intermediate revisions by 3 users not shown)
Line 4: Line 4:
  
  
−
=Topic=
+
=Introduction and Platforms=
  
−
The interest in autonomous vehicles is growing constantly, with lots of practical applications appearing on the marketplace and many more being actively studied in academia, industry and military research departments. Two of the most representative examples of such technology are unmanned aerial vehicles (UAVs) and autonomous driving systems.
+
The interest in autonomous vehicles is growing constantly, with lots of practical applications appearing on the marketplace and many more being actively studied in academia, industry, and military research departments. Two of the most representative examples of such technology are unmanned aerial vehicles (UAVs) and autonomous driving systems.
−
Autonomous systems are emerging in many fields in order to assist and help humans in a plethora of applications, like environment surveillance/inspection, etc. Fully autonomous driving is still out of reach, but virtually every major OEM (BMW, Volvo, Tesla Motors, GM) has a clear roadmap towards achieving this goal and is already commercializing advanced driver assistance systems (ADAS).
+
Autonomous systems are emerging in many fields in order to assist and help humans in a plethora of applications, like environment surveillance, inspection, monitoring, etc. Fully autonomous driving is still out of reach, but virtually every major OEM (BMW, Volvo, Tesla Motors, GM) has a clear roadmap towards achieving this goal and is already commercializing advanced driver assistance systems (ADAS). Autonomous vehicles represent a unique opportunity to effectively handle critical and expensive activities, compared to the traditional use of human operators only.
  
−
[[File:Taxonomy.png|thumb|left|350px|Rotorcraft UAVs taxonomy by vehicle calss-size.]]
+
When it comes to performing autonomous navigation, widely adopted tasks, such as obstacle avoidance, path planning, target/pedestrian detection, simultaneous localization, and mapping, etc., are very complex and computationally intensive. Given the high computational requirements that such workloads expose, current state-of-the-art solutions need quite powerful computational units. The typical class of devices used for such sophisticated algorithms are multi-cores high-end CPUs and embedded heterogeneous systems featuring powerful CPUs coupled with many-core accelerators (e.g., GPUs). In both classes, the order of magnitude for the power consumption is in the range of few Watts to tens of Watts.
  
−
Autonomous vehicles represent a unique opportunity to effectively handle critical and expensive activities, compared to the traditional use of human operators only. Autonomous UAVs are adopted for periodic tasks of inspection, like dyke inspection aimed to find cracks on the wall, nuclear plant control or cultivated field monitoring. They can also be extremely helpful in time-critical rescue missions, enhancing the response time and the effectiveness in a natural disaster scenario (e.g. earthquake, avalanche, etc.). ADAS systems are applied by car-makers in order to help the driver in the driving process, increasing the driving comfort, protecting the drivers and reducing accidents. It is clear how autonomous systems can improve the quality of such activities under several aspects: increasing the safety of human operators, reducing the monetary and time costs and improving the effectiveness of the mission (e.g. reduced inspection time or increase the safety on the drive).
+
Such computational requirements, and power budget, limit the applicability of such techniques to vehicles that expose enough power for the computation. If we consider for instance rotorcraft or flapping-wings UAVs, the available power budget for computation is between 5-20% of the overall power. Thus, reducing the power consumption of the control system in UAVs will become increasingly important as the size of the vehicle is scaled down. Indeed, reducing the UAV size quickly leads to an order-of-magnitude reduction of the power spent on the propellers. Besides, the computational load required to implement the UAV cognitive skills does not vary with the vehicle size, and will thus constitute an increasingly larger fraction of the total system power consumption.
  
−
To obtain such sophisticated and useful systems, advanced capabilities and cognitive skills are required. Depending on the specific use case, the desired autonomous navigation capability is reached combining one or more basic computational tasks. Widely adopted tasks, such as obstacle avoidance, path planning, target/pedestrian detection, simultaneous localization and mapping, etc, are very complex and computationally intensive. Given the high computational requirements that such workloads expose, current state-of-the-art solutions need quite powerful computational units. The typical class of devices used for such sophisticated algorithms are multi-cores high-end CPUs and embedded heterogeneous systems featuring powerful CPUs coupled with many-core accelerators (e.g. GPUs). In both classes, the order of magnitude for the power consumption is in the range of few Watts to tens of Watts.
+
===a. Pico-size Quadrotor (Fünfliber drone)===
 +
[[File:Fuenfliber.jpg|thumb|right|200px| The ''Fünfliber-drone'']]
  
−
[[File:Path_planner.png|thumb|right|450px|Path Planning application for autonomous driving.]]
+
The Fünfliber drone contains the Mr.Wolf SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: one, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. In contrast to the commercial GWT GAP8, Mr.Wolf features a Floating-Point Unit on the cluster. Furthermore, the drone features 6 Time of Flight sensors, a Himax camera, an optical flow sensor, a barometer, an IMU, an infrared camera and a radio chip.
  
−
Such computational requirements, and power budget, limit the applicability of such techniques to vehicles that expose enough power for the computation. If we consider for instance rotorcraft or flapping-wings UAVs, the available power budget for computation is in between 5-20% of the overall power. Thus, reducing the power consumption of the control system in UAVs will become increasingly important as the size of the vehicle is scaled down. Indeed, reducing the UAV size quickly leads to order-of-magnitude reduction of the power spent on the propellers. Beside, the computational load required to implement the UAV cognitive skills does not vary with the vehicle size, and will thus constitute an increasingly larger fraction of the total system power consumption.
 
  
−
Limiting the on-board power consumption could be achieved by streaming information from the vehicle (e.g. video stream) to either a base-station or the cloud in order to offload the computation, but this is rarely a viable option. In fact, we would introduce additional sever constraints (e.g. communication latency, channel noise, transmission power consumption, maximum distance to the base-station, etc.), limiting the actual autonomous navigation capability of the vehicle. For this reason all the most advanced autonomous navigation systems perform computation directly on-board of the vehicle.
+
.
  
−
[[File:UAV_DATE.png|thumb|left|300px|Hovering UAV with on-board sensors.]]
+
===b. Nano-size Quadrotor===
 +
[[File:cf_aideck.jpg|thumb|right|200px| The ''CrazyFlie 2.1'']]
  
−
These problems are further exacerbated if we consider the predominance of a trend towards device miniaturization. As already seen in many fields downsizing is the forthcoming technological improvement also in robotics, with a significant reduction of the vehicle dimensions. For example, commercial off the shelf nano rotor-crafts can have only 10 cm diameter and looking into the most extreme research areas we can already find insect-size flying robots. As a consequence of the reduced size of the vehicle we also have a reduction in term of battery dimension, on-board payload and available energy for computation. Also in the automotive field similar considerations in term of computational power budget still hold, where the interest of car-makers is in energy-efficient embedded solutions that directly affect the final cost for the customers.
+
Our main cyber-physical platform is the 27g nano-size ''Bitcraze CrazyFlie 2.1'' [1],. It has been used in many projects due to its dimension, versatility, and its open-source and open-hardware nature.
 +
Nowadays is well known how UAVs with high-level autonomous navigation capabilities are a hot topic both in industry and academia due to their numerous applications. However, autonomous navigation algorithms are demanding from a computational standpoint, and it is very challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter) because of the limited capabilities of their MCU-based controllers that are also in charge of running the control and estimation algorithms for flying the drone. In order to extend the computational capabilities of the drone, we equip it with the plug-in PC called the AI-Deck that enables complex artificial intelligence-based workloads to run onboard, with the possibility to achieve fully autonomous navigation capabilities.
  
−
Thus, it becomes clear that to keep pace with the ever-increasing demand for computation capabilities and the decreasing power budgets we have to target very energy efficient hardware/software solutions. On the other hands, accomplish such challenging task would allow to this new generation of robotic helpers to penetrate in the everyday life, paving the way also for new scenarios like intelligent tiny swarms of UAVs able to cooperate, as well as a fully autonomous driving car.
+
The AI-deck contains the GWT GAP8 SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: the first, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. Furthermore, the AI-deck also features an ultra-low-power onboard camera and a WiFi module.
  
 +
Our previous work presents a lightweight hardware-software solution based on a CNN that brings autonomous navigation on the Crazyflie+Ai-deck using only the onboard resources. Furthermore, we evaluated how the Parallel Ultra-Low-Power Platform [2] can enable running sophisticated machine learning algorithms for achieving autonomous navigation or localization capabilities.
  
−
==Nano-size Quadrotor==
+
[https://www.youtube.com/watch?v=VPeSnjoGDEc Demo Video]
−
[[File:CamUAV.jpg|thumb|left|250px| The ''CrazyFlie 2.0'' extended with camera and on-board computation for autonomous navigation. Developed at IIS by our former student Jaskirat Singh.]]
 
  
−
Our first cyberphysical platform is the nano-size ''Bitcraze CrazyFlie 2.0'' [1]. It has been used in many projects due to its dimension, versatility and its open-source and open-hardware nature.
 
−
Nowadays is well known how UAVs with high level autonomous navigation capabilities are a hot topic both in industry and academia due to their numerous applications. However, autonomous navigation algorithms are demanding from the computational standpoint, and it is very challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter) because of the limited capabilities of their MCU-based controllers. The nano-quadrotor is an appealing platform (among others) for addressing this challenging task.
 
−
In this context, we presented a lightweight hardware-software solution, bringing autonomous navigation on a commercial platform using only on-board computational resources. Furthermore, we evaluated how the Parallel Ultra Low-Power Platform [2] can enable the execution of even more sophisticated algorithms. [http://www.youtube.com/watch?v=T9fkjAp942A Demo Video]
 
  
−
==Nano-size Blimp==
 
−
One of our favorite nano-size platform is the IIS/TIK ''Nano-Blimp''.
 
−
A nano-sized blimp is a perfect candidate for long flight times because helium, a lighter-than-air gas, can provide lift and significantly reduce the energy requirements for flight.
 
  
−
[[File:Blimp1.png|thumb|left|400px|A) The self-sustainable nano-blimp developed at IIS/TIK by our student Kevin Keller. B) The blimp model with solar panel, MCU’s, battery, and rotor.]]
+
.
  
−
[[File:Blimp2.png|thumb|right|400px|A) The autonomous nano-blimp developed at IIS/TIK by our student Bence Szebedy. B) The blimp model with on-board camera, MCU’s, battery, and rotors for 3D movements.]]
+
=Topics=
 +
===a. Vision-based Autonomous Navigation===
  
−
In the first project of this series we introduced the nano-blimp. We demonstrated that, thanks to the helium-filled balloon, the energy requirement for hovering is significantly reduced.
+
[[File:drone_vicon_room.jpg|thumb|right|220px| The CrazyFlie equipped with the AI-deck performing autonomous navigation]]
−
Then, we extended the functionality of our first self-sustainable blimp prototype introducing additional motors and on-board camera, paving the way for autonomous navigation.
 
−
We enabled first horizontal movement creating a blimp that is able to move in three dimensions.
 
−
Then, we expanded the on-board processing capabilities with visual sensors and we incorporated, optimized, and improved a simple object tracking algorithm for autonomous flying nano-size UAVs.
 
  
−
==The PULP-Shield==
+
Vision-based perception algorithms traditionally employ simultaneous localization-and-mapping (SLAM), which is a technique that builds a 3D local map of the environment, which is used to plan the trajectory accordingly. Nevertheless, the main drawback of the approaches based on SLAM is that they are computationally demanding. Furthermore, while SLAM can be used for perception, it does not solve the challenging issue of inferring the control commands from the 3D map. More recent solutions, such as end-to-end convolutional neural networks (CNNs), mitigate this issue by directly estimating the optimal control
−
[[File:PULP-Shield.JPG|thumb|left|300px|The PULP-Shield coupled with Mr.Wolfe PULP chip.]]
+
commands, using camera images as input. Due to the recent advancement of low-power parallel hardware platforms such as PULP, convolutional neural networks (CNNs) can run very efficiently, even aboard a small nano-drone. Therefore, the GAP8 chip found on the AI-deck allows running complex NN models significantly faster than the classical single-core MCUs (such as the Cortex M4 found in Crazyflie).
−
Here at IIS we developed the PULP-Shield, the first pluggable PCB for extending the computational power on-board of the CrazyFlie 2.0 nano-size platform. Through this project we enabled the Parallel Ultra Low-Power Platform [2] to be the key computational unit to bring state-of-the-art complex vision algorithms for autonomous navigation into the nano-scale class of vehicles.
 
−
Thus, we enabled for the first time the ''Parallel Computational Paradigm'' on-board of this tiny class of vehicles. Even if the PULP architecture is meant to act as the main agent of the system, we aim to keep the pre-existing MCU (STM32) present on the UAV as the "coordinator" (i.e., host) of the system. Then, PULP can play the role of the accelerator in charge of performing the compute-intensive kernels.
 
−
Thus, embodying the classic paradigm Host + Accelerator, the goal is exploiting on one side the existing firmware running on the STM32 MCU, and on the other adding the high computational capability of the 8-cores chip in an ultra-low-power envelope.
 
−
Future directions for this project series are both in the implementation of state-of-the-art algorithms on-board of our nano-size platforms and on the design of a new cyberphysical system only based on the PULP architecture.
 
  
−
=Contact Information=
 
−
[[File:DP-UAV.jpeg|thumb|right|210px|[[:User:Dpalossi | Daniele Palossi]]]]
 
  
 +
.
 +
 +
===b. Lidar/Radar-based Autonomous Navigation===
 +
As an alternative to camera-based navigation methods, solutions based on distance estimation gained more and more popularity for accomplishing obstacle avoidance. This is mainly due to the technological advancement of optical time-of-flight (ToF) sensors. This class of sensors estimates the distance by emitting a light beam towards a target and measuring when the partially reflected beam arrives. Based on this, the sensor calculates the roundtrip time and determines the distance. These sensors were widely used in applications such as proximity sensing or presence detection. Moreover, they are also found in drones, where they enable the drone to estimate the height with respect to the ground for indoor applications. More recent solutions that employ this technology, include several such sensors grouped in a matrix, aiming to perform multi-object detection. Therefore, the output of such a sensor will be a low-resolution "image" that contains depth information. Compared to cameras, this approach has the advantage of working reliably in poor-light conditions.
 +
 +
===C. Indoor Localization with UWB===
 +
[[File:localization.jpg|thumb|right|320px| ]]
 +
 +
Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters.
 +
Estimation algorithms such as Kalman filters or multilateration (based on cost function optimization) are used to determine the position of the drone, given the anchor locations and the UWB measurements.
 +
b) presents the dual scenario, where the drone is used as a moving anchor and acquires range measurements in various spots to determine the location of a static, unknown position UWB node.
  
−
=====Daniele Palossi=====
+
However, there are still drawbacks associated with UWB: for instance, in some situations, physical factors (i.e., obstacles in the line-of-sight) or such antenna delays can significantly decrease the UWB ranging accuracy. Furthermore, since the ToA ranging is sequential, increasing the number of drones will result in fewer UWB measurements per drone (since the maximum ranging frequency is divided by the number of drones).
 +
Sophisticated learning algorithms can mitigate these issues, by modeling the UWB ranging errors and using the model to compensate for the errors.
 +
Moreover, learning algorithms can identify patterns in the UWB channel impulse response (CIR) and therefore enable features such as concurrent ranging and angle-of-arrival estimation.
  
−
* '''e-mail''': [mailto:dpalossi@iis.ee.ethz.ch dpalossi@iis.ee.ethz.ch]
+
The goal is to exploit the capabilities of the parallel chip (i.e., GAP8) found on the AI-deck to run algorithms that improve the localization accuracy in applications with drones and swarms.
−
* '''phone''': +41 44 633 88 43
 
−
* '''address''': Gloriastrasse 35, 8092 Zürich
 
−
* '''office''': ETZ J76.2
 
  
 +
=Contact Information=
  
−
<!--=====Hanna Mueller=====
+
=====Hanna Mueller=====
 +
[[File:hanna.jpg|thumb|right|150px|[[:User:Hanna | Hanna Mueller]]]]
  
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch hanmuell@iis.ee.ethz.ch]
 
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch hanmuell@iis.ee.ethz.ch]
Line 75: Line 75:
  
  
 +
[[File:bio_vlad2.JPG|thumb|right|150px|[[:User:Vladn | Vlad Niculescu]]]]
 
=====Vlad Niculescu=====
 
=====Vlad Niculescu=====
  
 
* '''e-mail''': [mailto:vladn@iis.ee.ethz.ch vladn@iis.ee.ethz.ch]
 
* '''e-mail''': [mailto:vladn@iis.ee.ethz.ch vladn@iis.ee.ethz.ch]
 
* '''address''': Gloriastrasse 35, 8092 Zürich
 
* '''address''': Gloriastrasse 35, 8092 Zürich
−
* '''office''': ETZ J68.1-->
+
* '''office''': ETZ J68.1
  
−
=Collaborations=
 
−
We are pleased to inform our students that we have the opportunity to offer co-supervised Master/Semester Thesis on the ''Autonomous UAVs'' topic in collaborations with other top-quality research groups like:
 
−
* '''TIK''': The Computer Engineering and Networks Laboratory - ETH Zürich - [http://www.tec.ee.ethz.ch/ Web Site]
 
−
* '''RPG''': Robotic and Perception Group - University of Zürich - [http://rpg.ifi.uzh.ch/ Web Site]
 
−
* '''MICREL''': Microelectronics Laboratory - University of Bologna - [http://www-micrel.deis.unibo.it/sitonew/ Web Site]
 
  
 +
[[File:DP21.png|thumb|right|150px|[[:User:Dpalossi | Dr. Daniele Palossi]]]]
 +
=====Dr. Daniele Palossi=====
  
 +
* '''e-mail''': [mailto:dpalossi@iis.ee.ethz.ch dpalossi@iis.ee.ethz.ch]
 +
* '''address''': Gloriastrasse 35, 8092 Zürich
 +
* '''office''': ETZ J65
 +
 +
=====Tommaso Polonelli=====
 +
[[File:tommaso.jpeg|thumb|right|150px|[[:User:Tommaso | Dr. Tommaso Polonelli]]]]
  
−
=Projects=
+
* '''e-mail''': [mailto:hanmuell@iis.ee.ethz.ch tommaso.polonelli@pbl.ee.ethz.ch]
−
We are listing a few projects below to give you an idea of what we do. However, we constantly have new project ideas and maybe some other approaches become obsolete in the very rapidly advancing research area. Please just contact us and come to talk with us.
+
* '''address''': Gloriastrasse 35, 8092 Zürich
 +
* '''office''': ETZ D97.5
  
−
==Available Projects==
 
−
<DynamicPageList>
 
−
category = Available
 
−
category = Digital
 
−
category = Energy Efficient Autonomous UAVs
 
−
suppresserrors=true
 
−
</DynamicPageList>
 
  
−
==Projects In Progress==
 
−
<DynamicPageList>
 
−
category = In progress
 
−
category = Digital
 
−
category = Energy Efficient Autonomous UAVs
 
−
suppresserrors=true
 
−
</DynamicPageList>
 
  
−
==Completed Projects==
+
==Projects Overview==
 +
===Available Projects===
 
<DynamicPageList>
 
<DynamicPageList>
−
category = Completed
+
category = Available
 
category = Digital
 
category = Digital
−
category = Energy Efficient Autonomous UAVs
+
category = UAV
 
suppresserrors=true
 
suppresserrors=true
 
</DynamicPageList>
 
</DynamicPageList>
Line 119: Line 111:
 
The group effort and the great contribution from the students of last few years has resulted in the following list of publications:
 
The group effort and the great contribution from the students of last few years has resulted in the following list of publications:
  
−
* '''2018''' - D. Palossi et Al., "Extending the Lifetime of Nano-Blimps via Dynamic Motor Control", ''Springer Journal of Signal Processing Systems (Springer JSPS)'', 2018 - [https://link.springer.com/article/10.1007/s11265-018-1343-1 On-line]
+
* '''2022''' - V. Niculescu et Al., "Fly, wake-up, find: UAV-based energy-efficient localization for distributed sensor nodes", ''Sustainable Computing: Informatics and Systems'', - [https://www.sciencedirect.com/science/article/pii/S2210537922000038 Link]
 +
 
 +
* '''2021''' - V. Niculescu et Al., "Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs", ''IEEE Journal on Emerging and Selected Topics in Circuits and Systems'', - [https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9606685 Link]
 +
 
 +
* '''2021''' - H. Mueller et Al., "Fünfiiber-Drone: A Modular Open-Platform 18-grams Autonomous Nano-Drone", ''2021 Design, Automation & Test in Europe Conference & Exhibition (DATE)'', - [https://ieeexplore.ieee.org/abstract/document/9474262 Link]
 +
 
 +
* '''2021''' - V. Niculescu et Al., "Automated Tuning of End-to-end Neural Flight Controllers for Autonomous Nano-drones", ''2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS)'', - [https://ieeexplore.ieee.org/abstract/document/9458550 Link]
 +
 
 +
* '''2020''' - V. Niculescu et Al., "An Energy-efficient Localization System for Imprecisely Positioned Sensor Nodes with Flying UAVs", ''2020 IEEE 18th International Conference on Industrial Informatics (INDIN)'', - [https://ieeexplore.ieee.org/abstract/document/9442075 Link]
  
 
* '''2017''' - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", ''7th IEEE International Workshop on Advances in Sensors and Interfaces'', June 15-16, Vieste, Italy, 2017 - [http://ieeexplore.ieee.org/document/7974242/ On-line]
 
* '''2017''' - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", ''7th IEEE International Workshop on Advances in Sensors and Interfaces'', June 15-16, Vieste, Italy, 2017 - [http://ieeexplore.ieee.org/document/7974242/ On-line]
−
 
−
* '''2017''' - B. Forsberg et Al., "GPU-Accelerated Real-Time Path Planning and the Predictable Execution Model", ''International Conference on Computational Science (ICCS)'', June 12-14, Zürich, Switzerland, 2017 - [http://www.sciencedirect.com/science/article/pii/S1877050917308256 On-line]
 
  
 
* '''2017''' - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", ''20th International Workshop on Software and Compilers for Embedded Systems (SCOPES)'', June 12-13, Sankt Goar, Germany, 2017 - [http://dl.acm.org/citation.cfm?id=3079072 On-line]
 
* '''2017''' - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", ''20th International Workshop on Software and Compilers for Embedded Systems (SCOPES)'', June 12-13, Sankt Goar, Germany, 2017 - [http://dl.acm.org/citation.cfm?id=3079072 On-line]
Line 134: Line 132:
  
 
* '''2016''' - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", ''2nd Workshop on Design of Low Power Embedded Systems (LP-EMS)'', May 16-18, Como, Italy, 2016 - [http://dl.acm.org/citation.cfm?id=2911712 On-line]
 
* '''2016''' - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", ''2nd Workshop on Design of Low Power Embedded Systems (LP-EMS)'', May 16-18, Como, Italy, 2016 - [http://dl.acm.org/citation.cfm?id=2911712 On-line]
−
 
−
* '''2016''' - F. Conti et Al., "Enabling the Heterogeneous Accelerator Model on Ultra-Low Power Microcontroller Platforms", ''Design, Automation and Test in Europe (DATE)'', March 14-18, Dresden, Germany, 2016 - [http://ieeexplore.ieee.org/document/7459494/ On-line]
 
−
 
−
 
  
 
=External Links=
 
=External Links=
−
* [http://www.bitcraze.io/crazyflie-2/ <nowiki> [1] Bitcraze CrazyFlie 2.0</nowiki>]
+
* [https://www.bitcraze.io/products/crazyflie-2-1/ <nowiki> [1] Bitcraze CrazyFlie 2.1</nowiki>]
 
* [http://www.pulp-platform.org/ <nowiki> [2] PULP Platform</nowiki>]
 
* [http://www.pulp-platform.org/ <nowiki> [2] PULP Platform</nowiki>]
  
  
 
[[#top|↑ top]]
 
[[#top|↑ top]]

Latest revision as of 15:02, 13 June 2022

A-B) The PULP-Shield PCB developed at IIS by our student Hanna Müller. C) Our nano-drone prototype based on the CrazyFlie 2.0 coupled with the PULP-Shield.


Introduction and Platforms

The interest in autonomous vehicles is growing constantly, with lots of practical applications appearing on the marketplace and many more being actively studied in academia, industry, and military research departments. Two of the most representative examples of such technology are unmanned aerial vehicles (UAVs) and autonomous driving systems. Autonomous systems are emerging in many fields in order to assist and help humans in a plethora of applications, like environment surveillance, inspection, monitoring, etc. Fully autonomous driving is still out of reach, but virtually every major OEM (BMW, Volvo, Tesla Motors, GM) has a clear roadmap towards achieving this goal and is already commercializing advanced driver assistance systems (ADAS). Autonomous vehicles represent a unique opportunity to effectively handle critical and expensive activities, compared to the traditional use of human operators only.

When it comes to performing autonomous navigation, widely adopted tasks, such as obstacle avoidance, path planning, target/pedestrian detection, simultaneous localization, and mapping, etc., are very complex and computationally intensive. Given the high computational requirements that such workloads expose, current state-of-the-art solutions need quite powerful computational units. The typical class of devices used for such sophisticated algorithms are multi-cores high-end CPUs and embedded heterogeneous systems featuring powerful CPUs coupled with many-core accelerators (e.g., GPUs). In both classes, the order of magnitude for the power consumption is in the range of few Watts to tens of Watts.

Such computational requirements, and power budget, limit the applicability of such techniques to vehicles that expose enough power for the computation. If we consider for instance rotorcraft or flapping-wings UAVs, the available power budget for computation is between 5-20% of the overall power. Thus, reducing the power consumption of the control system in UAVs will become increasingly important as the size of the vehicle is scaled down. Indeed, reducing the UAV size quickly leads to an order-of-magnitude reduction of the power spent on the propellers. Besides, the computational load required to implement the UAV cognitive skills does not vary with the vehicle size, and will thus constitute an increasingly larger fraction of the total system power consumption.

a. Pico-size Quadrotor (Fünfliber drone)

The Fünfliber-drone

The Fünfliber drone contains the Mr.Wolf SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: one, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. In contrast to the commercial GWT GAP8, Mr.Wolf features a Floating-Point Unit on the cluster. Furthermore, the drone features 6 Time of Flight sensors, a Himax camera, an optical flow sensor, a barometer, an IMU, an infrared camera and a radio chip.


.

b. Nano-size Quadrotor

The CrazyFlie 2.1

Our main cyber-physical platform is the 27g nano-size Bitcraze CrazyFlie 2.1 [1],. It has been used in many projects due to its dimension, versatility, and its open-source and open-hardware nature. Nowadays is well known how UAVs with high-level autonomous navigation capabilities are a hot topic both in industry and academia due to their numerous applications. However, autonomous navigation algorithms are demanding from a computational standpoint, and it is very challenging to run them on-board of nano-scale UAVs (i.e., few centimeters of diameter) because of the limited capabilities of their MCU-based controllers that are also in charge of running the control and estimation algorithms for flying the drone. In order to extend the computational capabilities of the drone, we equip it with the plug-in PC called the AI-Deck that enables complex artificial intelligence-based workloads to run onboard, with the possibility to achieve fully autonomous navigation capabilities.

The AI-deck contains the GWT GAP8 SoC based on the Parallel Ultra-Low-Power [2] architecture paradigm, which features 1+8 general-purpose RISC-Vcores: the first, named Fabric Controller (FC), acts as the central coordinator of the MCU; the remaining are grouped in a parallel Cluster (CL) - all cores are based on the open-source RI5CY design. Furthermore, the AI-deck also features an ultra-low-power onboard camera and a WiFi module.

Our previous work presents a lightweight hardware-software solution based on a CNN that brings autonomous navigation on the Crazyflie+Ai-deck using only the onboard resources. Furthermore, we evaluated how the Parallel Ultra-Low-Power Platform [2] can enable running sophisticated machine learning algorithms for achieving autonomous navigation or localization capabilities.

Demo Video


.

Topics

a. Vision-based Autonomous Navigation

The CrazyFlie equipped with the AI-deck performing autonomous navigation

Vision-based perception algorithms traditionally employ simultaneous localization-and-mapping (SLAM), which is a technique that builds a 3D local map of the environment, which is used to plan the trajectory accordingly. Nevertheless, the main drawback of the approaches based on SLAM is that they are computationally demanding. Furthermore, while SLAM can be used for perception, it does not solve the challenging issue of inferring the control commands from the 3D map. More recent solutions, such as end-to-end convolutional neural networks (CNNs), mitigate this issue by directly estimating the optimal control commands, using camera images as input. Due to the recent advancement of low-power parallel hardware platforms such as PULP, convolutional neural networks (CNNs) can run very efficiently, even aboard a small nano-drone. Therefore, the GAP8 chip found on the AI-deck allows running complex NN models significantly faster than the classical single-core MCUs (such as the Cortex M4 found in Crazyflie).


.

b. Lidar/Radar-based Autonomous Navigation

As an alternative to camera-based navigation methods, solutions based on distance estimation gained more and more popularity for accomplishing obstacle avoidance. This is mainly due to the technological advancement of optical time-of-flight (ToF) sensors. This class of sensors estimates the distance by emitting a light beam towards a target and measuring when the partially reflected beam arrives. Based on this, the sensor calculates the roundtrip time and determines the distance. These sensors were widely used in applications such as proximity sensing or presence detection. Moreover, they are also found in drones, where they enable the drone to estimate the height with respect to the ground for indoor applications. More recent solutions that employ this technology, include several such sensors grouped in a matrix, aiming to perform multi-object detection. Therefore, the output of such a sensor will be a low-resolution "image" that contains depth information. Compared to cameras, this approach has the advantage of working reliably in poor-light conditions.

C. Indoor Localization with UWB

Localization.jpg

Ultra-wideband (UWB) is one of the most promising and adopted ranging (i.e., distance measuring) technologies used for positioning and localization, as it enables centimeter-precision distance estimation and data transmission. In our applications, we use UWB with the time-of-arrival (ToA) technique, which determines the distance between two UWB nodes based on the travel time of a radio signal from the transmitter to the receiver. Due to its high-precision ranging, UWB enables range-based localization. a) shows a scenario where fixed UWB anchors with known positions enable the drone to determine its position using the range measurements from each anchor. Therefore, installing fixed UWB nodes in a room enables a drone/swarm of drones to localize themselves, follow certain trajectories, and perform indoor navigation. Due to the high precision of the UWB range measurements, the positioning error of the drones is on the order of tens of centimeters. Estimation algorithms such as Kalman filters or multilateration (based on cost function optimization) are used to determine the position of the drone, given the anchor locations and the UWB measurements. b) presents the dual scenario, where the drone is used as a moving anchor and acquires range measurements in various spots to determine the location of a static, unknown position UWB node.

However, there are still drawbacks associated with UWB: for instance, in some situations, physical factors (i.e., obstacles in the line-of-sight) or such antenna delays can significantly decrease the UWB ranging accuracy. Furthermore, since the ToA ranging is sequential, increasing the number of drones will result in fewer UWB measurements per drone (since the maximum ranging frequency is divided by the number of drones). Sophisticated learning algorithms can mitigate these issues, by modeling the UWB ranging errors and using the model to compensate for the errors. Moreover, learning algorithms can identify patterns in the UWB channel impulse response (CIR) and therefore enable features such as concurrent ranging and angle-of-arrival estimation.

The goal is to exploit the capabilities of the parallel chip (i.e., GAP8) found on the AI-deck to run algorithms that improve the localization accuracy in applications with drones and swarms.

Contact Information

Hanna Mueller
Hanna Mueller


Vlad Niculescu
Vlad Niculescu


Dr. Daniele Palossi
Dr. Daniele Palossi
Tommaso Polonelli
Dr. Tommaso Polonelli


Projects Overview

Available Projects


Publications

The group effort and the great contribution from the students of last few years has resulted in the following list of publications:

  • 2022 - V. Niculescu et Al., "Fly, wake-up, find: UAV-based energy-efficient localization for distributed sensor nodes", Sustainable Computing: Informatics and Systems, - Link
  • 2021 - V. Niculescu et Al., "Improving Autonomous Nano-drones Performance via Automated End-to-End Optimization and Deployment of DNNs", IEEE Journal on Emerging and Selected Topics in Circuits and Systems, - Link
  • 2021 - H. Mueller et Al., "Fünfiiber-Drone: A Modular Open-Platform 18-grams Autonomous Nano-Drone", 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), - Link
  • 2021 - V. Niculescu et Al., "Automated Tuning of End-to-end Neural Flight Controllers for Autonomous Nano-drones", 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), - Link
  • 2020 - V. Niculescu et Al., "An Energy-efficient Localization System for Imprecisely Positioned Sensor Nodes with Flying UAVs", 2020 IEEE 18th International Conference on Industrial Informatics (INDIN), - Link
  • 2017 - D. Palossi et Al., "Target Following on Nano-Scale Unmanned Aerial Vehicles", 7th IEEE International Workshop on Advances in Sensors and Interfaces, June 15-16, Vieste, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "On the Accuracy of Near-Optimal CPU-Based Path Planning for UAVs", 20th International Workshop on Software and Compilers for Embedded Systems (SCOPES), June 12-13, Sankt Goar, Germany, 2017 - On-line
  • 2017 - D. Palossi et Al., "Self-sustainability in Nano Unmanned Aerial Vehicles: A Blimp Case Study", Computing Frontiers (CF), May 15-17, Siena, Italy, 2017 - On-line
  • 2017 - D. Palossi et Al., "Ultra Low-Power Visual Odometry for Nano-Scale Unmanned Aerial Vehicles", Design, Automation and Test in Europe (DATE), March 27-31, Lausanne, Switzerland, 2017 - On-line
  • 2016 - D. Palossi et Al., "Exploring Single Source Shortest Path Parallelization on Shared Memory Accelerator", 19th International Workshop on Software and Compilers for Embedded Systems (SCOPES), May 23-25, Sankt Goar, Germany, 2016 - On-line
  • 2016 - D. Palossi et Al., "An Energy-Efficient Parallel Algorithm for Real-Time Near-Optimal UAV Path Planning", 2nd Workshop on Design of Low Power Embedded Systems (LP-EMS), May 16-18, Como, Italy, 2016 - On-line

External Links


↑ top

Retrieved from "http://iis-projects.ee.ethz.ch/index.php?title=Energy_Efficient_Autonomous_UAVs&oldid=7826"