Difference between revisions of "GPT on the edge"
From iis-projects
Gislamoglu (talk | contribs) |
(→Status: Available) |
||
| Line 31: | Line 31: | ||
* 20% Measurements and validation | * 20% Measurements and validation | ||
| − | == Status: | + | == Status: Taken == |
* Type: Semester or Master Thesis (multiple students possible) | * Type: Semester or Master Thesis (multiple students possible) | ||
| Line 65: | Line 65: | ||
* Currently involved students: | * Currently involved students: | ||
| − | ** | + | ** Linus Joos |
[[Category:Available]] [[Category:Digital]] [[Category:Event-Driven Computing]] [[Category:Deep Learning Projects]] [[Category:EmbeddedAI]] [[Category:SmartSensors]] [[Category:System Design]] [[Category:2023]] [[Category:2024]] [[Category:Semester Thesis]] [[Category:Master Thesis]] [[Category:Julian]] [[Category:Mayerph]] [[Category:Vivianep]] [[Category:Hot]] | [[Category:Available]] [[Category:Digital]] [[Category:Event-Driven Computing]] [[Category:Deep Learning Projects]] [[Category:EmbeddedAI]] [[Category:SmartSensors]] [[Category:System Design]] [[Category:2023]] [[Category:2024]] [[Category:Semester Thesis]] [[Category:Master Thesis]] [[Category:Julian]] [[Category:Mayerph]] [[Category:Vivianep]] [[Category:Hot]] | ||
Latest revision as of 11:35, 12 March 2024
Contents
Overview
This thesis explores the development and implementation of compact AI models on microcontrollers, aiming to democratize advanced AI by reducing reliance on cloud computing and enhancing the efficiency and privacy of edge-deployed systems. It tackles the challenge of preserving the models' knowledge retention while significantly minimizing their size and energy consumption.
Project description
The advent of large language models like GPT and Llama has revolutionized natural language processing. However, their deployment is predominantly cloud-based, requiring substantial computational resources. This thesis proposal targets an innovative approach to enable full functionality of these models on the very edge itself, i.e. on microcontrollers. This shall be achieved by leveraging dedicated processing acceleration and memory support. The shift from cloud-based AI systems to on-device edge-deployable AI algorithms aims to make advanced AI technologies more accessible and responsive while minimizing latency, reducing bandwidth requirements, and enhancing data privacy. Moreover, the shift towards utilizing highly energy-efficient devices circumvents the need for extensive infrastructure, thereby decreasing energy consumption. Crucially, this endeavor to miniaturize model sizes for microcontroller compatibility presents the challenge of developing new AI algorithms that maintain robust knowledge retention even as model dimensions diminish.
Your task in this project will be one or several of the tasks mentioned below. Depending on your thesis (Semester/Master thesis), tasks will be assigned according to your interests and skills.
Tasks:
- Study on requirements and quantization techniques
- Porting of open-source pre-trained transformer
- Implementation of a suitable interface
Prerequisites (not all needed!) depending of Tasks
- Embedded Firmware Design and experience in C-Code
- Experience in Machine Learning
- Parallel programming
Type of work
- 20% Literature study
- 60% Software design
- 20% Measurements and validation
Status: Taken
- Type: Semester or Master Thesis (multiple students possible)
- Professor: : Prof. Dr. Luca Benini
- Supervisors:
 |
Julian Moosmann
|
 |
Viviane Potocnik
|
 |
Philipp Mayer
|
- Currently involved students:
- Linus Joos
