Personal tools

Evaluating An Ultra low Power Vision Node

From iis-projects

Revision as of 08:02, 24 November 2020 by Scheremo (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


With the on-going surge of interest in bringing machine learning to wearables and the edge of computing devices, novel solutions to reducing the energy consumption of such devices on the system-level are required. One of the key ideas in event-driven computing is the reduction of power consumption by only triggering computations when events occur and entering an idle mode when there are no events. This can be used in the context of machine learning to only trigger network inference when events are registered. Qualcommm has presented an ultra-low power vision platform at TinyML 2019 (Youtube Video), which we have been able to get our hands on.

Project description

In this project, a novel event-driven camera by Qualcomm will be evaluated and charaterized using their development kit, including an ARM Cortex-M Series microcontroller. After initial efforts to measure power consumption, latency and assuring functionality of the sensor platform and depending on the students interests, a neural network for object detection can be trained and deployed to the platform. A full demo, including a smart wake-up trigger to minimize power consumption can be targeted.

The student is required to:

  1. Read up on documentation provided by Qualcomm
  2. Measure the key parameters of the sensor platform
  3. Prepare a demo of the camera's working principle

Depending on the progress, the following points may be investigated:

  1. Implementation of a Bluetooth interface
  2. Training and deployment of an energy-proportional CNN onto the Cortex-M device

Required Skills

  • Basic knowledge of the C language and embedded system programming

Skills you might find useful:

  • Machine learning with Python
  • Neural network deployment on embedded devices (Machine Learning on Microcontrollers)
  • Previous experience with the Bluetooth 5 LE stack


Luca Benini
Status: Available

Possible to complete as a Master, Semester or Bachelor Thesis

Supervision: Moritz Scherer, Alfio Di Mauro

Meetings & Presentations

The students and advisor(s) agree on weekly meetings to discuss all relevant decisions and decide on how to proceed. Of course, additional meetings can be organized to address urgent issues. At the end of the project, you have to present/defend your work during a 15 min. presentation and 5 min. of discussion as part of the IIS colloquium.