Microsoft Azure has announced itself as the first cloud service provider to offer AMD Instinct MI300X accelerator chips for rent on virtual machines. The AMD Instinct MI300X is a speed accelerator card that solely features GPU CDNA 3 (though there is a version, MI300A, that includes both CPU and GPU). One of its key highlights is the use of high-speed HBM3 memory, which significantly increases bandwidth compared to previous models.
The Microsoft VM, known as ND MI300X v5, utilizes 8 MI300X cards with a massive 1.5TB memory and a memory bandwidth of 5.3 TB/s, making it ideal for large-scale AI processing tasks. The network system uses InfiniBand with a speed of 400 Gb/s per GPU, totaling 3.2 Tb/s per VM.
The software side fully supports AMD’s ROCm platform, allowing popular frameworks like TensorFlow and PyTorch to be deployed seamlessly, along with other Microsoft AI libraries such as ONNX Runtime, DeepSpeed, and MSCCL.
Microsoft also mentions that the VMs have been tailored for running popular models like GPT-4 Turbo and various open-source models.
TLDR: Microsoft Azure offers AMD Instinct MI300X accelerator chips on VMs for high-speed AI processing with extensive memory and bandwidth capabilities. The platform supports popular AI frameworks and custom models.
Leave a Comment