View source for Benchmarking RISC-V-based Accelerator Cards for Inference (multiple SA)

<!-- Benchmarking RISC-V-based Accelerator Cards for Inference (multiple SA) -->

[[Category:Digital]]
[[Category:High Performance SoCs]]
[[Category:2023]]
[[Category:Semester Thesis]]
[[Category:Hot]]
[[Category:Cykoenig]]
[[Category:Available]]


= Overview =

== Status: Available ==

* Type: Semester thesis (multiple students possible)
* Professor: Prof. Dr. L. Benini
* Supervisors:
** [[:User:Cykoenig | Cyril Koenig]]: [mailto:cykoenig@iis.ee.ethz.ch cykoenig@iis.ee.ethz.ch]

= Introduction =

In the recent years, multiple companies have adopted the RISC-V ISA to propose high-quality chips ready for production. These Systems-on-Chip have evolved from Embedded MCUs a few years ago, to high performance CPUs and computing cards today.

At the Integrated Systems Laboratory (IIS), we port great interest in these new architecture and the open source tools around them.

The Sophon SG2042 [1] is a 64-cores CPU based on the 64-bits Xuantie C620 cores. This RISC-V CPU boots Linux and is used in the Milk-V Pioneer server. This server features all the required PC-like interfaces to replace a desktop computer, or integrate into an high-performance cluster. The motherboard contains two PCIe slots to hold computing cards, as GPU, network cards, or specific accelerators. Finally, SG2042 comes with an open source bootloader, Fedora Linux image, and GCC based compiler included THead extensions.

[[File:Pioneer.jpg|400px|]] [2]

The Tenstorrent Grayskull e75 [2] is a RISC-V based PCIe accelerator card. It focuses on Machine Learning inference with its 96 Tensix cores (each containing 5 RISC-V processors, a tensor accelerator, a vector co-processor and up to 1.5MB of SRAM).

The Axelera AI Metis AIPU [3] is a RISC-V based accelerator also targeting ML inference. It is divided in four AI cores based on leveraging in memory computing dataflows. The dataflow targets matrix vector multiplications. The Metis AIPU is embedded in a PCIe accelerator card.


= Project description =

In this project, you will be executing and benchmarking ML inference, e.g., ResNet-50, on Milk-V servers. You will then select one of the PCIe accelerator card to accelerate this workload.

You are expected to acquire a good comprehension of the architectures cited to justify bench-marking results. If applicable, you will rely on the different SDKs to try and run Large Language Models.


== Character ==

* 20% Literature/architecture review
* 60% Programming
* 20% Evaluation

== Prerequisites ==

* Strong interest in computer architecture
* Experience in C/Python programming
* Preferred: Knowledge or prior experience with RISC-V

= References =

[https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf](https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf)

[https://milkv.io/docs/pioneer/](https://milkv.io/docs/pioneer/)

[https://github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)