Report: Apple wants to run its AI locally on your iPhone, iPad, Mac, and even Apple Watch

Apple has always paid great attention to the privacy and security of its users. In that manner, a lot of features on an average iPhone are local by design, as some sensitive user data never leaves their devices. The company wants to continue that practice with its new AI tools, as a new report from The Information reveals new details about what Apple might reveal during the upcoming WWDC.

Apple’s AI strategy centers around the existing deal with Google. Apple is using a large version of Google's Gemini model to train a smaller, distilled version capable of running locally on iPhones, Macs, and even Apple Watches. The idea is to use Apple's custom silicon, specifically the Neural Processing Units built into recent devices, to handle everyday AI tasks locally, rather than sending queries to the cloud.

This means Apple devices will run a model similar in nature to current open-source AI models, except it won't actually be open-source. But it carries the main benefit that open-source models offer, and that is data privacy, since user interactions with AI happen entirely on-device. Going this route also means Apple avoids pouring additional billions into its own AI data centers, a trend that's been hard to miss among Big Tech lately.

Not every AI interaction stays local, though. The full Gemini model is too large for Apple's own Private Cloud Compute infrastructure, so tasks that need more reasoning will still be sent to and processed in Google Cloud.

The obvious question is how capable a distilled on-device model will actually be. Running a small AI model locally on a phone is already possible today, but these models are barely capable of performing anything other than the most basic tasks.

Still, Apple's version should be more capable than a typical 2B parameter model, since it will be purpose-built and optimized specifically for Apple hardware. On top of that, Apple is said to be eyeing startups that specialize in shrinking models for local hardware, with Liquid AI, a Cambridge, Massachusetts firm focused on on-device inference, mentioned as one it has considered.

WWDC kicks off on June 8.

Source: The Information