Cerebras Systems, a company founded by former AMD employees, has unveiled their latest chip model, the WSE-3. This chip boasts an unprecedented 4 trillion transistors, 900,000 AI processing cores, on-chip 44GB SRAM, and AI processing performance of 125 petaflops.
The Cerebras WSE-3 chip can be externally linked to up to 1.3PB of memory, enabling training of future models larger than GPT-4 or Gemini by a factor of 10 (up to 24 trillion parameters stored in one memory unit without partitioning).
Its competitor, the NVIDIA H100 chip, a current market favorite, pales in comparison to the Cerebras WSE-3. Cerebras’ strategy focuses on creating significantly larger chips than GPUs and utilizing a single chip instead of multiple GPUs (WSE-3 chip area is 57 times larger than H100, with over 52 times more cores).
The WSE-3 chip will be integrated into the Cerebras CS-3 supercomputer, which can scale up to 2048 clusters and complete training of the Llama 70B model in a single day. The CS-3 machine also achieves energy savings with a 2x performance increase compared to its predecessor.
TLDR: Cerebras Systems introduces the powerful WSE-3 chip with unparalleled processing capabilities, surpassing current market leaders and driving advancements in AI processing technology.
Leave a Comment