Google has just launched LiteRT.js, a new library that enables machine learning models to run locally within the web browser, bypassing the need for server-side processing. The firm said this will bring native AI performance to web browsers via its mobile-focused LiteRT runtime, though LiteRT.js also works on desktops.
The new library uses WebAssembly and hardware acceleration such as WebGPU and WebNN to replace the slower TensorFlow.js, which uses a slower JavaScript-based kernel.
The LiteRT runtime has been reserved for Android and iOS until now. With today’s update, Google is exposing the runtime via WebAssembly to turn the browser into a more capable platform for AI and ML workloads.
The search giant claims that this new runtime delivers 3x greater speeds over existing solutions on current hardware. Specifically, it was tested on a 2024 Apple MacBook Pro with M4 Silicon. In the real world, for users on older hardware or using browsers with different engines, performance could vary significantly.
For developers looking to switch over from TensorFlow.js, the process is straightforward. If you already have a .tflite file, you just need to switch your JavaScript runtime to LiteRT.js. However, if you have a TensorFlow/Keras SavedModel, then you can use the LiteRT Converter built into the Python TensorFlow package. You can read more about the conversion process on Google for Developers.
Going forward, it will be interesting to see whether Google eventually sunsets or de-prioritizes TensorFlow.js in favor of LiteRT.js. It will also be interesting to see how it really performs on other hardware besides a MacBook.
Source: Google