Intel Unveils Gaudi 3, a Revolutionary AI Accelerator Chip with 4x Improved Performance, Ready to Take on H100

Intel has unveiled their latest artificial intelligence accelerator chip, Intel Gaudi 3, which was previously showcased last year. The chip utilizes a 5-nanometer manufacturing process and originated from the acquisition of Habana Labs in 2019. Upgraded to Gaudi 2 in 2022, this chip is positioned by Intel to compete directly with NVIDIA’s top-tier chips.

The architecture of Gaudi 3 consists of three main processing units within the chip:

– 64 Tensor Processor Cores (TPC) for primary processing
– 8 Matrix Multiplication Engines (MME) for parallel matrix processing
– 24 Networking Interface Cards (NIC) with network processing units for Ethernet traffic of 200 Gb

Intel has opted for standard Ethernet connections instead of proprietary networking ports, similar to NVIDIA’s approach. Additionally, Gaudi 3 boasts 128GB of HBMe2 memory with a combined bandwidth of 3.7TB. This design enables efficient data processing on fewer chips, improving performance and reducing data center costs. The card also supports expansion through PCIe add-ins for specialized tasks like fine-tuning and retrieval-augmented generation (RAG).

Comparing Gaudi 3 to Gaudi 2, Intel reports a fourfold increase in AI floating-point performance, a 1.5-fold increase in memory bandwidth, and a twofold increase in network bandwidth. When pitted against its competitor, NVIDIA H100, Intel claims Gaudi 3 offers faster model training times and over 50% improvement in inference throughput.

Gaudi 3 is slated to be available through server OEM manufacturers in the second quarter of 2024, with partners like Dell, Hewlett Packard Enterprise, Lenovo, and Supermicro already confirmed. General availability is expected in the third quarter of the same year.

TLDR:
Intel introduces the Intel Gaudi 3 chip for AI processing, featuring advanced architecture and improved performance metrics compared to its predecessor and competitors. Scheduled for release in 2024 through server OEMs, Gaudi 3 aims to enhance data processing efficiency and reduce costs in data centers.