Design of a High-performance Hybrid PTZ for Multimodal Vision Systems
Dynamic Vision Sensors (DVS) or also called Event-based cameras can detect (when stationary placed) fast-moving and small objects and open-up tons of new possibilities for AI and tinyML. We are creating a completely new system, with an autonomous base station and distributed smart sensor nodes to run cutting-edge AI algorithms and perform novel sensor fusion techniques.
Modern Pan-Tilt-Zoom (PTZ) Units are built to host normal Coupled Charged Device (CCD) Sensors or CMOS sensors and as such commercial Cameras. However, a new promising camera sensor technology called dynamic vision sensor (DVS) or event camera, captures not the light intensity of each pixel, instead it captures the intensity change. As such the data created is a time-stamp with the pixel coordinate and the pixel’s change in intensity and moving objects are detectable, while static objects are not seen. This event-driven approach opens up new fields for detection, especially for fast-moving, small objects. Nevertheless, if the sensor is moved, the camera records a lot of changes and creates overwhelming many events. To overcome this issue, we are targeting to create a novel high-performance PTZ Unit which can create a non-moving stationary environment for a very short amount of time to acquire event-based data in that time-fragment and continue moving step-wise if needed.
Your task in this project will be one or several out of the tasks mentioned below. Depending on your thesis (Semester/Master thesis), tasks will be assigned accordingly to your interests and skills.
- E-CAD design (Altium) of PTZ-Unit
- M-CAD design (Fusion360 or Siemens NX) of PTZ-Unit
- Embedded Firmware Design for the PTZ-Unit
Prerequisites (not all needed!) depending of Tasks
- Embedded Firmware Design and experience in Free RTOS, Zephyr, etc…
- C-Code programming
- Mechanical/Circuit design tools (e.g. Altium)
- Experience in ML on MCU or deep knowledge of ML and strong will to deploy on the edge
Type of work
- 20% Literature study
- 60% Software and/or Hardware design
- 20% Measurements and validation
- Type: Semester or Master Thesis (multiple students possible)
- Professor: : Prof. Dr. Luca Benini
- Currently involved students: