Microsoft has announced support for running the DeepSeek-R1 model in distilled sizes of 1.5B, 7B, and 14B parameters on the NPU chip of the Copilot+ PC by PC Group, as previously stated.
The company utilizes various techniques such as 4-bit block-wise quantization, QuaRot, and ONNX QdQ to reduce the model size, techniques that were first introduced in their Phi Silica model.
The advantage of running models on NPU is cost efficiency in cloud rental fees, as newer PCs come equipped with NPU for improved model processing compared to traditional CPUs and GPUs.
Compatibility will start with the Snapdragon X series PCs, followed by Intel Core Ultra 200V and AMD Ryzen in the next phase. A minimum spec requirement includes 16GB RAM and access through the AI Toolkit for Visual Studio Code.
TLDR: Microsoft supports running DeepSeek-R1 model on NPU for cost-effective model processing, using advanced quantization techniques for model size reduction and starting compatibility with select PC models.
Leave a Comment