Difference between revisions of "Benchmarking RISC-V-based Accelerator Cards for Inference (multiple SA)"
From iis-projects
| Line 29: | Line 29: | ||
[[File:Pioneer.jpg|200px|]] [2] | [[File:Pioneer.jpg|200px|]] [2] | ||
| − | The Tenstorrent Grayskull e75 [3] is a RISC-V based PCIe accelerator card. It focuses on Machine Learning inference with its 96 Tensix cores (each containing 5 RISC-V processors, a tensor accelerator, a vector co-processor and up to 1.5MB of SRAM). | + | The Tenstorrent Grayskull e75 [3] is a RISC-V based PCIe accelerator card. It focuses on Machine Learning inference with its 96 Tensix cores (each containing 5 RISC-V processors, a tensor accelerator, a vector co-processor and up to 1.5MB of SRAM). The Grayskull card comes with two open source SDKs, TT-Buda for off-the-shelf inference, and TT-Metallium for fine grained programmability. |
[[File:E75.png|200px|]] [3] | [[File:E75.png|200px|]] [3] | ||
| − | The Axelera AI Metis AIPU [4] is a RISC-V based accelerator also targeting ML inference. It is divided in four AI cores based on leveraging in memory computing dataflows. The dataflow targets matrix vector multiplications. The Metis AIPU is embedded in a PCIe accelerator card. | + | The Axelera AI Metis AIPU [4] is a RISC-V based accelerator also targeting ML inference. It is divided in four AI cores based on leveraging in memory computing dataflows. The dataflow targets matrix vector multiplications. The Metis AIPU is embedded in a PCIe accelerator card. Metis relies on the Voyager SDK to run off-the-shelf models. |
[[File:Metis.jpg|200px|]] [4] | [[File:Metis.jpg|200px|]] [4] | ||
Revision as of 09:40, 12 April 2024
Contents
Overview
Status: Available
- Type: Semester thesis (multiple students possible)
- Professor: Prof. Dr. L. Benini
- Supervisors:
Introduction
In the recent years, multiple companies have adopted the RISC-V ISA to propose high-quality chips ready for production. These Systems-on-Chip have evolved from Embedded MCUs a few years ago, to high performance CPUs and computing cards today.
At the Integrated Systems Laboratory (IIS), we port great interest in these new architecture and the open source tools around them.
The Sophon SG2042 [1] is a 64-cores CPU based on the 64-bits Xuantie C620 cores. This RISC-V CPU boots Linux and is used in the Milk-V Pioneer server. This server features all the required PC-like interfaces to replace a desktop computer, or integrate into an high-performance cluster. The motherboard contains two PCIe slots to hold computing cards, as GPU, network cards, or specific accelerators. Finally, SG2042 comes with an open source bootloader, Fedora Linux image, and GCC based compiler included THead extensions.
[2]
The Tenstorrent Grayskull e75 [3] is a RISC-V based PCIe accelerator card. It focuses on Machine Learning inference with its 96 Tensix cores (each containing 5 RISC-V processors, a tensor accelerator, a vector co-processor and up to 1.5MB of SRAM). The Grayskull card comes with two open source SDKs, TT-Buda for off-the-shelf inference, and TT-Metallium for fine grained programmability.
[3]
The Axelera AI Metis AIPU [4] is a RISC-V based accelerator also targeting ML inference. It is divided in four AI cores based on leveraging in memory computing dataflows. The dataflow targets matrix vector multiplications. The Metis AIPU is embedded in a PCIe accelerator card. Metis relies on the Voyager SDK to run off-the-shelf models.
[4]
Project description
In this project, you will be executing and benchmarking ML inference, e.g., ResNet-50, on Milk-V servers. You will then select one of the PCIe accelerator card to accelerate this workload.
You are expected to acquire a good comprehension of the architectures cited to justify bench-marking results. If applicable, you will rely on the different SDKs to try and run Large Language Models.
Character
- 20% Literature/architecture review
- 60% Programming
- 20% Evaluation
Prerequisites
- Strong interest in computer architecture
- Experience in C/Python programming
- Preferred: Knowledge or prior experience with RISC-V
References
[1](https://github.com/milkv-pioneer/pioneer-files/blob/main/hardware/SG2042-TRM.pdf)
[2](https://milkv.io/docs/pioneer/)
