Benchmarking a RISC-V-based Server on LLMs/Foundation Models (SA or MA)
From iis-projects
Contents
Overview
Status: Available
- Type: Semester or Master Thesis (multiple students possible)
- Professor: Prof. Dr. L. Benini
- Supervisors:
Introduction
Milk-V is a company committed to delivering high-quality RISC-V products to developers, enterprises, and consumers. It focuses on the development of both hardware and software ecosystems around the RISC-V architecture. Milk-V strongly supports open-source initiatives and aims to enrich the RISC-V product landscape, hoping that through its efforts and those of the community, the future of RISC-V products will be as vast and luminous as the Milky Way.
The Milk-V Pioneer is a developer motherboard utilizing the SOPHON SG2042 [1], designed in the standard microATX (mATX) form factor. It offers PC-like interfaces and compatibility with PC industrial standards, aiming to provide a native RISC-V development environment and desktop experience. The Pioneer is targeted at RISC-V developers and hardware pioneers, offering a platform to engage with cutting-edge RISC-V technology. This motherboard serves as an excellent choice for those interested in exploring and developing within the RISC-V architecture.
[2]
Project description
In this project, you will be executing LLMs and Foundation Models, e.g., Whisper AI, to Milk-V servers and benchmark their performance.
You will first select a framework to execute LLMs in C/C++, for instance llama.cpp [3]. You will then evaluate one or several models using this framework on the SG2042 CPU. Finally, you will identify potential limitations or improvements of the code related to the microarchitecture.
Character
- 20% Literature/architecture review
- 60% Programming
- 20% Evaluation
Prerequisites
- Strong interest in computer architecture
- Experience in C programming
- Preferred: Knowledge or prior experience with RISC-V
References
[1](https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf)
