Event-Driven Vision on an embedded platform
With the on-going surge of interest in bringing machine learning to wearables and the edge of computing devices, novel solutions to reducing the energy consumption of such devices on the system-level are required. One of the key ideas in event-driven computing is the reduction of power consumption by only triggering computations when events occur and entering an idle mode when there are no events. This can be used in the context of machine learning to only trigger network inference when events are registered. Previous work at IIS and Fondazione Bruno Kessler (FBK) has led to a prototype sensor board, which includes an FPGA for control of an event-driven imaging sensor and data storage as well as an SPI and CPI interface to connect to a microcontroller.
In this project, a novel event-driven camera by FBK will be deployed and characterized in conjunction with a GAP 8 processor to enable energy-proportional image recognition on embedded platforms. Depending on the student’s interests and progress, a full working sensor node including Bluetooth communication and deployment of a Convolutional Neural Network (CNN) can be implemented.
The student is required to:
- Study the event-driven sensor and prototype board developed at IIS
- Implement and verify HDL code to exchange data between the camera sensor and the microcontroller
- Connect the sensor board to a GAPuino evaluation board and evaluate the solution in terms of power consumption, latency and throughput
Depending on the progress, the following points may be investigated:
- Implementation of a Bluetooth interface
- Training and deployment of an energy-proportional CNN onto GAP 8
- Basic knowledge of the System Verilog or VHDL language and circuit design (VLSI 1)
- Basic knowledge of the C language and embedded system programming
Skills you might find useful:
- Experience with FPGA toolchains
- PCB design with Altium
- Machine learning with Python
- Neural network deployment on embedded devices (Machine Learning on Microcontrollers)
Possible to complete as a Master, Semester or Bachelor Thesis
Meetings & Presentations
The students and advisor(s) agree on weekly meetings to discuss all relevant decisions and decide on how to proceed. Of course, additional meetings can be organized to address urgent issues. At the end of the project, you have to present/defend your work during a 15 min. presentation and 5 min. of discussion as part of the IIS colloquium.