Integration Of A Smart Vision System

Introduction

Vision-based surveillance systems and the data collected by them are important technological building blocks for military and civilian applications. Existing camera surveillance solutions come with severe drawbacks; Surveillance cameras require power in the order of Watts, which precludes them from being used in a portable system. Furthermore, surveillance system typically do not come with any localized intelligence, so their recorded data is streamed to a central cloud server, where it is processed. While this approach leads to accurate classification of the streamed data, it also poses a security risk, since surveillance data is highly sensitive.

Project description

In this project, a novel, distributed and energy-efficient surveillance system will be brought up and optimized by the student. Our system consists of battery-powered, LoRa-based IoT-class vision nodes, which perform near-field surveillance without streaming sensitive data. Once these vision nodes detect activity in their field of view, they send an alarm to a centralized high-performance vision platform, which is able to pan, tilt and zoom its field of view to the origin of the alarm. In summary, this system reduces the overall energy cost of surveillance and privacy of subjects in the surveilled area, since no camera data is transferred.

The student will optimize the prototype system in the following system:

1. Evaluation and optimization of system response latency
2. Integration and optimization of Pan-Tilt Zoom control of the centralized vision node
3. Preparation of a demonstration of the system

Depending on the progress, the following points may be investigated:

1. Optimization & Deployment of embedded Person Detection algorithm
2. Optimization of embedded IoT-node power consumption

Required Skills

• Basic knowledge of the C language and embedded system programming

Skills you might find useful:

• Machine learning with Python

• Neural network deployment on embedded devices (Machine Learning on Microcontrollers)

• Previous experience with the LoRaWAN protocol and stack

Professor

Luca Benini

Status: Available

Possible to complete as a Master, Semester or Bachelor Thesis

Supervision: Moritz Scherer scheremo@iis.ee.ethz.ch, Michele Magno magnom@pbl.ee.ethz.ch

Meetings & Presentations

The students and advisor(s) agree on weekly meetings to discuss all relevant decisions and decide on how to proceed. Of course, additional meetings can be organized to address urgent issues. At the end of the project, you have to present/defend your work during a 15 min. presentation and 5 min. of discussion as part of the IIS colloquium.