DeepSeek conquered the cell world and it’s now increasing to Home windows – with the complete help of Microsoft, surprisingly. Yesterday, the software program big added the DeepSeek R1 mannequin to its Azure AI Foundry to permit builders to check and construct cloud-based apps and providers with it. In the present day, Microsoft introduced that it’s bringing distilled variations of R1 to Copilot+ PCs.
The distilled fashions will first be out there to gadgets powered by Snapdragon X chips, those with Intel Core Extremely 200V processors after which AMD Ryzen AI 9 primarily based PCs.
The primary mannequin can be DeepSeek-R1-Distill-Qwen-1.5B (i.e. a 1.5 billion parameter mannequin) with bigger and extra succesful 7B and 14B fashions coming quickly. These can be out there for obtain from Microsoft’s AI Toolkit.
Microsoft needed to tweak these fashions to optimize them to run on gadgets with NPUs. Operations that rely closely on reminiscence entry run on the CPU, whereas computationally-intensive operations just like the transformer block run on the NPU. With the optimizations, Microsoft managed to attain quick time to first token (130ms) and a throughput charge of 16 tokens per second for brief prompts (below 64 tokens). Notice {that a} “token” is just like a vowel (importantly, one token is normally a couple of character lengthy).
Microsoft is a powerful supporter of and deeply invested in OpenAI (the makers of ChatGPT and GPT-4o), however it appears that evidently it doesn’t play favorites – its Azure Playground has GPT fashions (OpenAI), Llama (Meta), Mistral (an AI firm), now DeepSeek too.
DeepSeek R1 within the Azure AI Foundry playground
Anyway, should you’re extra into native AI, obtain the AI Toolkit for VS Code first. From there, you need to be capable to obtain the mannequin regionally (e.g. “deepseek_r1_1_5” is the 1.5B mannequin). Lastly, hit Attempt in Playground and see how sensible this distilled model of R1 is.
“Model distillation”, typically referred to as “knowledge distillation”, is the method of taking a big AI mannequin (the complete DeepSeek R1 has 671 billion parameters) and transferring as a lot of its data as potential to a smaller mannequin (e.g. 1.5 billion parameters). It’s not an ideal course of and the distilled mannequin is much less succesful than the complete mannequin – however its smaller measurement permits it to run instantly on client {hardware} (as a substitute of devoted AI {hardware} that prices tens of 1000’s of {dollars}).
Supply
