Last year, Microsoft revealed Copilot+ PCs featuring a dedicated Neural Processing Unit (NPU) capable of over 40 TOPS or more. These Copilot+ PCs came with Phi-Silica, an on-device SLM to bring language intelligence capabilities to Microsoft's own first-party apps and apps from other 3rd-party developers.
Today, Microsoft revealed Mu, a new on-device small language model built into Windows 11. Microsoft's goal was to create an AI-powered agent within the Settings app that can understand a user's natural language queries and integrate it into the existing search box for a smooth user experience. Hence, Microsoft is using the new Mu model to power the new agent feature in the Settings app, which was made available to Windows Insiders recently in the Dev Channel with Copilot+ PCs.
Like Phi-Silica, Mu is designed to operate efficiently on NPUs, delivering over 100 tokens per second while running locally. In the official blog post, Microsoft explained how they designed and trained the Mu language model. Here's an overview of the Mu language model:
- Mu is a 330M encoder-decoder language model.
- The encoder-decoder approach achieved about 47% lower first-token latency and 4.7× higher decoding speed compared to a decoder-only model of similar size.
- Mu uses weight sharing in certain components to reduce the total parameter count.
- Microsoft trained Mu using NVIDIA A100 GPUs on Azure Machine Learning.
- Mu is nearly comparable in performance to a similarly fine-tuned Phi-3.5-mini, despite being one-tenth of the size.
- The model is better suited for multi-word queries. So for short or partial-word inputs, the Settings app will continue to surface lexical and semantic search results in the search box.
Interested users can download Windows 11 Build 26120.3964 (KB5058496) or higher to check out this new AI-powered agent in the Windows Settings app.
_small.jpg)
5 Comments - Add comment