NVIDIA has unveiled the Hopper H200 data center GPU, which is an upgrade from the previously released H100 in 2022.
The standout feature of the NVIDIA H200 is its use of the new HBM3e memory, which is faster than its predecessor. It can deliver data at a rate of 4.8 TBps with a memory size of 141GB, giving it a bandwidth increase of 2.4 times compared to the memory in the previous NVIDIA A100.
The H200 GPU will be used with the NVIDIA HGX H200 board, which supports both 4/8 GPU configurations and is compatible with the existing HGX H100 system. This allows for easy upgrades for organizations that already have the previous hardware.
NVIDIA claims that the H200 has improved performance for running Llama 2 models, achieving nearly double the speed of the H100. As for the 8-GPU HGX H200 server, it has a greater FP8 computational performance of over 32 petaflops.
The H200 will begin shipping in the second quarter of 2024. Leading cloud providers such as AWS, GCP, Azure, and Oracle Cloud have already placed their orders, while various server manufacturers are preparing to sell the product according to the schedule.
In addition, the Grace Hopper GH200 chip, which combines the Grace CPU and Hopper GPU, has been confirmed to be included with the H200. Previously, it was only mentioned as being a Hopper architecture GPU that uses HBM3e memory. It is also set to be available for purchase in the second quarter of 2024.
TL;DR: NVIDIA has introduced the Hopper H200, an upgraded data center GPU that features faster HBM3e memory, increased bandwidth, and improved performance for running complex models. It will be available for purchase in the second quarter of 2024, with leading cloud providers and server manufacturers already showing interest. Additionally, the H200 includes the Grace Hopper GH200 chip, combining CPU and GPU capabilities in a single package.