Difference between revisions of "Deep Learning Projects"
From iis-projects
m (→Projects) |
(→Projects) |
||
| Line 11: | Line 11: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
| − | ! Project Name !! Description !! Platform !! Workload Type !! Contact(s) | + | ! Status !! Project Name !! Description !! Platform !! Workload Type !! Contact(s) |
|- | |- | ||
| − | | CBinfer for Speech Recognition || We have recently published an approach to dramatically reduce computation effort when performing object detection on video streams with limited frame-to-frame changes (cf. [https://arxiv.org/pdf/1704.04313.pdf paper]). We think this approach could also be applied to audio signals for continuous listening to void commands: when looking at MFCCs or the short-term Fourier transform, changes in the spectrum between neighboring time windows are also limited. || Embedded GPU (Tegra X2) || SW (GPU, algo evals) || [[:User:lukasc|Lukas Cavigelli]] | + | | taken || CBinfer for Speech Recognition || We have recently published an approach to dramatically reduce computation effort when performing object detection on video streams with limited frame-to-frame changes (cf. [https://arxiv.org/pdf/1704.04313.pdf paper]). We think this approach could also be applied to audio signals for continuous listening to void commands: when looking at MFCCs or the short-term Fourier transform, changes in the spectrum between neighboring time windows are also limited. || Embedded GPU (Tegra X2) || SW (GPU, algo evals) || [[:User:lukasc|Lukas Cavigelli]] |
|- | |- | ||
<!--- | CBinfer+ || Running CNNs to a large extent means a tremendous computational effort, particularly when looking at object detection and image segmentation. For many applications we want to run everything in real-time on video data. We exploit that only few pixels change from frame to frame (cf. [https://arxiv.org/pdf/1704.04313.pdf paper]). We have several ideas on a) how to improve the algorithm, and b) would like to optimize the implementation for the next generation of (embedded) GPUs. You can be part of this (already successful) research path. || Embedded GPU (Tegra X2) || SW (GPU, algo evals) || [[:User:lukasc|Lukas Cavigelli]] | <!--- | CBinfer+ || Running CNNs to a large extent means a tremendous computational effort, particularly when looking at object detection and image segmentation. For many applications we want to run everything in real-time on video data. We exploit that only few pixels change from frame to frame (cf. [https://arxiv.org/pdf/1704.04313.pdf paper]). We have several ideas on a) how to improve the algorithm, and b) would like to optimize the implementation for the next generation of (embedded) GPUs. You can be part of this (already successful) research path. || Embedded GPU (Tegra X2) || SW (GPU, algo evals) || [[:User:lukasc|Lukas Cavigelli]] | ||
|- | |- | ||
---> | ---> | ||
| − | | CBinfer HW || The motiviation here is the same as for CBinfer+. Here we want to build a custom hardware architecture for ASICs capable of running this type of workload. || ASIC || HW (ASIC) || [[:User:lukasc|Lukas Cavigelli]] | + | | available || CBinfer HW || The motiviation here is the same as for CBinfer+. Here we want to build a custom hardware architecture for ASICs capable of running this type of workload. || ASIC || HW (ASIC) || [[:User:lukasc|Lukas Cavigelli]] |
|- | |- | ||
| − | | One-shot/Few-shot Learning || One-shot learning comes in handy whenever it is not possible to collect a large dataset. Consider for example face identification as a form of opening you apartment's door, where the user provides a single picture (not 100s) and is recognized reliably from then on. In this project you would apply a method called Prototypical Networks (cf. [[https://arxiv.org/abs/1703.05175 paper], [https://github.com/jakesnell/prototypical-networks code]]) to learn to identify faces. Once you have trained such a DNN, you will optimize it for an embedded system to run it in real time. For a master thesis, an interesting additional step could be to look at expanding this further to share information between multiple nodes/cameras and learn to re-identify faces also as they evolve over time. || Embedded GPU or Microcontroller || SW (algo, uC) || [[:User:lukasc|Lukas Cavigelli]], [[:User:andrire|Renzo Andri]] | + | | taken || One-shot/Few-shot Learning || One-shot learning comes in handy whenever it is not possible to collect a large dataset. Consider for example face identification as a form of opening you apartment's door, where the user provides a single picture (not 100s) and is recognized reliably from then on. In this project you would apply a method called Prototypical Networks (cf. [[https://arxiv.org/abs/1703.05175 paper], [https://github.com/jakesnell/prototypical-networks code]]) to learn to identify faces. Once you have trained such a DNN, you will optimize it for an embedded system to run it in real time. For a master thesis, an interesting additional step could be to look at expanding this further to share information between multiple nodes/cameras and learn to re-identify faces also as they evolve over time. || Embedded GPU or Microcontroller || SW (algo, uC) || [[:User:lukasc|Lukas Cavigelli]], [[:User:andrire|Renzo Andri]] |
|- | |- | ||
| − | | Distributed/Federated Learning || With the increasing number of IoT devices equipped with a bunch of sensor, it is not feasible to always stream all the data back to a server. Therefore, there is the need to learn on the node itself and synchronize/merge the network in a periodic scheme. || Embedded GPU || SW(algo, evals) || [[:User:andrire|Renzo Andri]], [[:User:lukasc|Lukas Cavigelli]] | + | | available || Distributed/Federated Learning || With the increasing number of IoT devices equipped with a bunch of sensor, it is not feasible to always stream all the data back to a server. Therefore, there is the need to learn on the node itself and synchronize/merge the network in a periodic scheme. || Embedded GPU || SW(algo, evals) || [[:User:andrire|Renzo Andri]], [[:User:lukasc|Lukas Cavigelli]] |
|- | |- | ||
| − | | Self-Learning Drone || Autonomous Driving is a hot topic nowadays, but also self-learning approaches (i.e. re-inforcement learning) have had a big success (e.g. AlphaGo from Google beat the world champion in Go. We want a drone to learn from its environment such that the drone is able to solve a task independantly. || ML frameworks (e.g. Torch)/GPU, Drone Simulation (ROS/Gazebo) || SW (Training) || [[:User:andrire|Renzo Andri]], [[:User:dpalossi|Daniele Palossi]] | + | | available || Self-Learning Drone || Autonomous Driving is a hot topic nowadays, but also self-learning approaches (i.e. re-inforcement learning) have had a big success (e.g. AlphaGo from Google beat the world champion in Go. We want a drone to learn from its environment such that the drone is able to solve a task independantly. || ML frameworks (e.g. Torch)/GPU, Drone Simulation (ROS/Gazebo) || SW (Training) || [[:User:andrire|Renzo Andri]], [[:User:dpalossi|Daniele Palossi]] |
|- | |- | ||
| − | | On-chip learning || Neural Networks are compute and resource intensive and are usually run on power-intensive GPU clusters, but we would like to exploit them also on the everywhere IoT devices. To reach that we need to develop new hardware architecture optimized for this application. This also include to check new algorithmic approach, which can reduce the compute or memory footprint of these networks. || ASIC || HW (ASIC) || [[:User:andrire|Renzo Andri]] | + | | available || On-chip learning || Neural Networks are compute and resource intensive and are usually run on power-intensive GPU clusters, but we would like to exploit them also on the everywhere IoT devices. To reach that we need to develop new hardware architecture optimized for this application. This also include to check new algorithmic approach, which can reduce the compute or memory footprint of these networks. || ASIC || HW (ASIC) || [[:User:andrire|Renzo Andri]] |
|- | |- | ||
| − | | The Always-On Self-Learning Camera Node || Example || Example || Example | + | | available || SAR data analysis || You will explore different data analysis options for unsupervised learning to segment SAR radar images. || Workstation|| SW (algo evals) || [[:User:lukasc|Lukas Cavigelli]] |
| + | |- | ||
| + | | -- || The Always-On Self-Learning Camera Node || Example || Example || Example | ||
|} | |} | ||
Revision as of 11:56, 19 February 2018
We are listing a few projects below to give you an idea of what we do. However, we constantly have new project ideas and maybe some other approaches become obsolete in the very rapidly advancing research area. Please just contact the people of a project most similar to what you would like to do, and come talk to us.
Prerequisites
We have no strict, general requirements, as they are highly dependent on the exact project steps. The projects will be adapted to the skills and interests of the student(s) -- just come talk to us! If you don't know about GPU programming or CNNs or ... just let us know and we can together determine what is a useful way to go -- after all you are here to learn not only about project work, but also to develop your technical skills.
Only hard requirements:
- Excitement for deep learning
- For VLSI projects: VLSI 1 or equivalent
Projects
| Status | Project Name | Description | Platform | Workload Type | Contact(s) |
|---|---|---|---|---|---|
| taken | CBinfer for Speech Recognition | We have recently published an approach to dramatically reduce computation effort when performing object detection on video streams with limited frame-to-frame changes (cf. paper). We think this approach could also be applied to audio signals for continuous listening to void commands: when looking at MFCCs or the short-term Fourier transform, changes in the spectrum between neighboring time windows are also limited. | Embedded GPU (Tegra X2) | SW (GPU, algo evals) | Lukas Cavigelli |
| available | CBinfer HW | The motiviation here is the same as for CBinfer+. Here we want to build a custom hardware architecture for ASICs capable of running this type of workload. | ASIC | HW (ASIC) | Lukas Cavigelli |
| taken | One-shot/Few-shot Learning | One-shot learning comes in handy whenever it is not possible to collect a large dataset. Consider for example face identification as a form of opening you apartment's door, where the user provides a single picture (not 100s) and is recognized reliably from then on. In this project you would apply a method called Prototypical Networks (cf. [paper, code]) to learn to identify faces. Once you have trained such a DNN, you will optimize it for an embedded system to run it in real time. For a master thesis, an interesting additional step could be to look at expanding this further to share information between multiple nodes/cameras and learn to re-identify faces also as they evolve over time. | Embedded GPU or Microcontroller | SW (algo, uC) | Lukas Cavigelli, Renzo Andri |
| available | Distributed/Federated Learning | With the increasing number of IoT devices equipped with a bunch of sensor, it is not feasible to always stream all the data back to a server. Therefore, there is the need to learn on the node itself and synchronize/merge the network in a periodic scheme. | Embedded GPU | SW(algo, evals) | Renzo Andri, Lukas Cavigelli |
| available | Self-Learning Drone | Autonomous Driving is a hot topic nowadays, but also self-learning approaches (i.e. re-inforcement learning) have had a big success (e.g. AlphaGo from Google beat the world champion in Go. We want a drone to learn from its environment such that the drone is able to solve a task independantly. | ML frameworks (e.g. Torch)/GPU, Drone Simulation (ROS/Gazebo) | SW (Training) | Renzo Andri, Daniele Palossi |
| available | On-chip learning | Neural Networks are compute and resource intensive and are usually run on power-intensive GPU clusters, but we would like to exploit them also on the everywhere IoT devices. To reach that we need to develop new hardware architecture optimized for this application. This also include to check new algorithmic approach, which can reduce the compute or memory footprint of these networks. | ASIC | HW (ASIC) | Renzo Andri |
| available | SAR data analysis | You will explore different data analysis options for unsupervised learning to segment SAR radar images. | Workstation | SW (algo evals) | Lukas Cavigelli |
| -- | The Always-On Self-Learning Camera Node | Example | Example | Example |
Workload types: SW (GPU), SW (microcontr.), SW (algorithm evals), HW (FPGA), HW (ASIC), HW (PCB)
Where to find us
Renzo Andri, ETZ J 76.2, andrire@iis.ee.ethz.ch
Lukas Cavigelli, ETZ J 76.2, cavigelli@iis.ee.ethz.ch
