Today, Apple has unveiled the Mac Studio, featuring the new M3 Ultra chip for high performance. A noteworthy highlight from this launch is Apple showcasing the speed of running the LLM AI model, a first for the M3 Ultra. It claims to run LLM up to 16.9 times faster than the M1 Ultra and boasts a staggering 512GB of RAM (larger than the M2 Ultra’s 192GB), enabling the running of multi-billion parameter models.
The competitiveness of Apple Silicon chips lies in their unified memory architecture between the CPU and GPU. With a large amount of RAM, they can handle large LLM models efficiently. Developers have favored Apple Silicon for its performance advantage.
Loading LLM tasks faces a bottleneck in memory bandwidth, as it involves running large binary-like workloads that require swapping data in and out of the processing unit. Apple’s Ultra-class chips have significantly high memory bandwidth, giving them a competitive edge. In a recent test, the M2 Ultra with 192GB of RAM managed to run Llama 3 at a speed of 145.82 token/s. While Apple did not disclose the specific model used for testing, it utilized LM Studio 0.3.9 and ran a model with a size of 174.63GB, larger than what the M1 Ultra with 128GB RAM was tested with. Comparing to the M2 Ultra with 192GB of RAM, the speed increase is surprisingly high, likely due to the memory bandwidth of the M3 Ultra (819GB/s) being close to the M2 Ultra (800GB/s).
Another advantage of the M3 Ultra is the inclusion of Thunderbolt 5 with a bandwidth of 120Gb/s, three times higher than Thunderbolt 4 in the M2 Ultra. This allows for larger cluster configurations.
The market for computers specifically for running LLM models has been steadily gaining popularity, especially for programs that require high-performance LLM. Following suit, even NVIDIA had to release Project DIGITS offering up to 128GB of RAM for a hefty $3,000 or Framework PCs utilizing the AMD Ryzen AI Max+ 395 chip with 128GB of RAM for $1,999. The Mac Studio M3 Ultra with 512GB of RAM comes at a price of 332,400 baht, considered quite reasonable for those in need of powerhouse computers for running large models.
TLDR: Apple introduces the Mac Studio with the powerful M3 Ultra chip, boasting high LLM performance and a massive 512GB of RAM, making it a competitive choice in the market for running large AI models.
Leave a Comment