AMD has begun delivering its Instinct MI300 chip, which was announced earlier this year. The chip is divided into two sub-models: the MI300A, which is an APU with an integrated CPU, and the MI300X, which is purely an accelerator chip.
The MI300A comes with 128GB of HBM3 memory and focuses on improving energy efficiency by 1.9 times compared to the previous MI250X model.
As for the MI300X, it is a CDNA 3 architecture accelerator chip, offering a 40% increase in processing units and a 1.7 times memory bandwidth expansion. With 192GB of HBM3 memory and support for FP8 data, it can run the LLaMA-2 70B model in a single chip. This is especially useful for organizations that need to run LLMs internally.
Apart from these two chips, AMD has also released the ROCm 6 SDK, which provides additional functions in the LLM group, such as FlahAttention, HIPGraph, and vLLM. In recent times, more LLM developers have been turning to AMD, such as Lamini and MosaicML.
In response to this announcement, cloud service providers and server manufacturers have expressed their interest in adopting the MI300 chips for their services. One particularly surprising company, Meta, has stated that it has already started using the MI300X with ROCm 6.
TLDR: AMD has started delivering the Instinct MI300 chips, including the MI300A APU and the MI300X accelerator chip. The MI300A focuses on energy efficiency, while the MI300X offers increased processing units and memory bandwidth. AMD has also released the ROCm 6 SDK, which provides additional functions for developing LLMs. Various cloud service providers and server manufacturers have expressed interest in adopting the MI300 chips, including Meta, which has already started using the MI300X with ROCm 6.
Leave a Comment