Difference between revisions of "Benchmarking a RISC-V-based Server on LLMs/Foundation Models (SA or MA)"
From iis-projects
| (3 intermediate revisions by 2 users not shown) | |||
| Line 20: | Line 20: | ||
** [[:User:Xiaywang | Xiaying Wang]]: [mailto:xiaywang@iis.ee.ethz.ch xiaywang@iis.ee.ethz.ch] | ** [[:User:Xiaywang | Xiaying Wang]]: [mailto:xiaywang@iis.ee.ethz.ch xiaywang@iis.ee.ethz.ch] | ||
** [[:User:Cykoenig | Cyril Koenig]]: [mailto:cykoenig@iis.ee.ethz.ch cykoenig@iis.ee.ethz.ch] | ** [[:User:Cykoenig | Cyril Koenig]]: [mailto:cykoenig@iis.ee.ethz.ch cykoenig@iis.ee.ethz.ch] | ||
| + | ** [[:User: Vivianep | Viviane Potocnik]]: [mailto:vivianep@iis.ee.ethz.ch vivianep@iis.ee.ethz.ch] | ||
= Introduction = | = Introduction = | ||
| Line 25: | Line 26: | ||
Milk-V is a company committed to delivering high-quality RISC-V products to developers, enterprises, and consumers. It focuses on the development of both hardware and software ecosystems around the RISC-V architecture. Milk-V strongly supports open-source initiatives and aims to enrich the RISC-V product landscape, hoping that through its efforts and those of the community, the future of RISC-V products will be as vast and luminous as the Milky Way. | Milk-V is a company committed to delivering high-quality RISC-V products to developers, enterprises, and consumers. It focuses on the development of both hardware and software ecosystems around the RISC-V architecture. Milk-V strongly supports open-source initiatives and aims to enrich the RISC-V product landscape, hoping that through its efforts and those of the community, the future of RISC-V products will be as vast and luminous as the Milky Way. | ||
| − | The Milk-V Pioneer is a developer motherboard utilizing the SOPHON SG2042, designed in the standard microATX (mATX) form factor. It offers PC-like interfaces and compatibility with PC industrial standards, aiming to provide a native RISC-V development environment and desktop experience. The Pioneer is targeted at RISC-V developers and hardware pioneers, offering a platform to engage with cutting-edge RISC-V technology. This motherboard serves as an excellent choice for those interested in exploring and developing within the RISC-V architecture. | + | The Milk-V Pioneer is a developer motherboard utilizing the SOPHON SG2042 [1], designed in the standard microATX (mATX) form factor. It offers PC-like interfaces and compatibility with PC industrial standards, aiming to provide a native RISC-V development environment and desktop experience. The Pioneer is targeted at RISC-V developers and hardware pioneers, offering a platform to engage with cutting-edge RISC-V technology. This motherboard serves as an excellent choice for those interested in exploring and developing within the RISC-V architecture. |
| − | [[File:Pioneer.jpg| | + | [[File:Pioneer.jpg|400px|]] [2] |
= Project description = | = Project description = | ||
In this project, you will be executing LLMs and Foundation Models, e.g., Whisper AI, to Milk-V servers and benchmark their performance. | In this project, you will be executing LLMs and Foundation Models, e.g., Whisper AI, to Milk-V servers and benchmark their performance. | ||
| + | |||
| + | You will first select a framework to execute LLMs in C/C++, for instance llama.cpp [3]. You will then evaluate one or several models using this framework on the SG2042 CPU. Finally, you will identify potential limitations or improvements of the code related to the microarchitecture. | ||
== Character == | == Character == | ||
| Line 47: | Line 50: | ||
= References = | = References = | ||
| − | [ | + | [https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf](https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf) |
| + | |||
| + | [https://milkv.io/docs/pioneer/](https://milkv.io/docs/pioneer/) | ||
| + | |||
| + | [https://github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp) | ||
Latest revision as of 11:37, 12 March 2024
Contents
Overview
Status: Available
- Type: Semester or Master Thesis (multiple students possible)
- Professor: Prof. Dr. L. Benini
- Supervisors:
Introduction
Milk-V is a company committed to delivering high-quality RISC-V products to developers, enterprises, and consumers. It focuses on the development of both hardware and software ecosystems around the RISC-V architecture. Milk-V strongly supports open-source initiatives and aims to enrich the RISC-V product landscape, hoping that through its efforts and those of the community, the future of RISC-V products will be as vast and luminous as the Milky Way.
The Milk-V Pioneer is a developer motherboard utilizing the SOPHON SG2042 [1], designed in the standard microATX (mATX) form factor. It offers PC-like interfaces and compatibility with PC industrial standards, aiming to provide a native RISC-V development environment and desktop experience. The Pioneer is targeted at RISC-V developers and hardware pioneers, offering a platform to engage with cutting-edge RISC-V technology. This motherboard serves as an excellent choice for those interested in exploring and developing within the RISC-V architecture.
[2]
Project description
In this project, you will be executing LLMs and Foundation Models, e.g., Whisper AI, to Milk-V servers and benchmark their performance.
You will first select a framework to execute LLMs in C/C++, for instance llama.cpp [3]. You will then evaluate one or several models using this framework on the SG2042 CPU. Finally, you will identify potential limitations or improvements of the code related to the microarchitecture.
Character
- 20% Literature/architecture review
- 60% Programming
- 20% Evaluation
Prerequisites
- Strong interest in computer architecture
- Experience in C programming
- Preferred: Knowledge or prior experience with RISC-V
References
[1](https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf)
