The Novel Battle of Cutting-Edge Graphics Cards: AMD Triumphs Over NVIDIA, Dueling to Determine the Alpha AI Model’s Rapidity

Back in the day, we were familiar with this type of battle in benchmarking gaming graphics cards. However, as the main market shifted towards AI-accelerating GPUs, this kind of competition has resurfaced in a new domain.

Just last week, AMD started selling their Instinct MI300X AI processing chips, and there’s an interesting point to note. Lisa Su, CEO of AMD, showcased impressive numbers that overshadowed NVIDIA’s H100. Using the popular Llama-2-70B model, AMD’s chip proved to be 1.2 times (for a single GPU) and 1.4 times (for an 8-GPU server) more powerful.

NVIDIA couldn’t accept this direct challenge, so they responded through their company blog, claiming that it was an inaccurate benchmark. They stated that AMD chose software that wasn’t specifically optimized for the H100 GPUs (NVIDIA openly released it as TensorRT-LLM). If the correct software and unbiased metrics were used, the H100’s performance would surpass AMD’s by more than 2 times.

But the battle doesn’t end there. AMD fought back by accusing NVIDIA of intentionally distorting the benchmarking process. They pointed out that NVIDIA solely used their own TensorRT-LLM software (while AMD used vLLM), compared different levels of decimal points (FP16 for AMD, FP8 for NVIDIA), and manipulated AMD’s latency/throughput numbers to their advantage.

AMD made it clear that the numbers showcased on stage were benchmark results from November, and they have since come up with new numbers using the improved ROCm software. The performance has been enhanced, leading to AMD’s victory being even more apparent. With vLLM benchmarking (over 2.1 times more powerful than NVIDIA), or even if measuring with NVIDIA’s methods (TensorRT-LLM vs vLLM), AMD still triumphed by 1.3 times. Additionally, when comparing different decimal point usage (FP8 vs FP16), AMD’s latency values remained superior. All in all, they emerged victorious in three areas.

TLDR: AMD introduced their Instinct MI300X chip, which outperformed NVIDIA’s H100 in benchmarking. NVIDIA claimed it was an unfair benchmark, but AMD responded, proving their victory even further.