Google has released a new app on GitHub called AI Edge Gallery. It’s an experimental Android app that’s not even in the Google Play Store yet for running generative AI large language models offline. The application will also be coming to iOS devices in the near future.
What is AI Edge Gallery?
Unlike Google’s other AI app, Gemini, the AI Edge Gallery is all about running AI models you’ve downloaded locally on your device. The processing is all performed locally, so you get more privacy and offline functionality. With Gemini and ChatGPT, you need to be connected to the net for them to respond and all of your requests are processed on Google and OpenAI servers - so they’re not the most privacy-friendly.
When you install the app, you get brought to a home screen with “Ask Image”, “Prompt Lab”, and “AI Chat” options. Clicking on any of those will take you through to another screen where you can pick a model to run your query through. The Ask Image mode lets you add an image and type a prompt to ask a question about it. The Prompt Lab has Free form, Rewrite tone, Summarize text, and Code snippet tabs to perform those actions. Finally, AI Chat lets you type back and forth with the AI models.
The four models that are included out of the box (though you need to download them individually, Hugging Face account also required) are:
- Gemma-3n-E2B-it-int4
- Gemma-3n-E4B-it-int4
- Gemma3-1B-IT q4
- QWEN2.5-1.5B-Instruct q8
It’s also possible to add extra models by pressing the + in the lower-left of the home screen and importing a local model. Depending on the model and your hardware, the app may use your GPU or CPU to process responses to your queries; anything processed with a GPU will be faster.
Who is it for?
Tech enthusiasts looking to run the newest AI models in a private environment will appreciate AI Edge Gallery. Out of all the offline AI apps that I’ve tried on Android, this one feels the most polished, it feels like any other Google app. Developers may also find this tool useful for testing LiteRT .task models. While it’s very cool that you can use the models offline, processing responses on your own device can be processor intensive, so you may see your battery drain a bit faster than normal if you ask it much.
AI Edge Gallery is able to perform well on your local device using a special engine developed by Google called LiteRT, or Lite Runtime. It’s optimized software that allows AI features to work quickly locally, without having to send all your data off to the cloud.
Things to be aware of
While it’s nice to run AI models offline, you should be aware that doing so on your limited hardware will mean slower performance than a powerful server in Google’s cloud. Another consideration is the size of the LLMs which you will download. The default models range in size from 560 MB to 4.4 GB, and they will live in your phone’s storage area until you decide to delete them, so you’ll want to make sure you have enough room.
It should also be noted that the iOS version of this app is not yet available. Also, if you’re on Android, right now, you must get the app via an APK file on GitHub, rather than through the more-trusted Google Play Store. Hopefully it arrives on both stores in the future for greater reach.
It will certainly be interesting seeing which way Google takes offline AI in the future. It’d certainly be nice to have a screen in the Settings app that lets you load in offline models and for you to invoke them similar to how Gemini is invoked now. In the meantime, we can use Google’s AI Edge Gallery. You can find out more on GitHub.
Update: Mentioned the Hugging Face requirement to download models.
1 Comment - Add comment