LiteRT：オンデバイスAIのためのユニバーサルフレームワーク

LiteRT: The Universal Framework for On-Device AI

Since we first introduced LiteRT in 2024, we have focused on evolving our ML tech stack from its TensorFlow Lite (TFLite) foundation into a modern on-device AI framework. While TFLite set the standard for classical ML, our mission is to empower developers to deploy today’s cutting-edge AI on-device just as seamlessly as they integrated classical ML in the past.

At Google I/O ‘25, we shared a preview of this evolution: a high-performance runtime designed specifically for advanced hardware acceleration. Today, we are excited to announce that these advanced acceleration capabilities have fully graduated into the LiteRT production stack, available now for all developers.

This milestone solidifies LiteRT as the universal on-device inference framework for the AI era, representing a significant leap over TFLite for being:

Faster: delivers 1.4x faster GPU performance than TFLite, and introduces new, state-of-the-art NPU acceleration.

Simpler: provides a unified, streamlined workflow for GPU and NPU acceleration across edge platforms.

Powerful: supports superior cross-platform GenAI deployment for popular open models like Gemma.

Flexible: offers first-class PyTorch/JAX support via seamless model conversion.

All of this is delivered while maintaining the same reliable, cross-platform deployment you trust since TFLite.

Here is how LiteRT empowers you in building the next-generation of on-device AI.

High-performance cross-platform GPU acceleration

Moving beyond the initial GPU acceleration on Android announced at I/O ‘25, we are excited to introduce the full, comprehensive GPU support across Android, iOS, macOS, Windows, Linux, and Web. This expansion provides developers with a reliable, high-performance acceleration option that scales significantly beyond classical CPU inference.

LiteRT maximizes the reach by introducing robust support for OpenCL, OpenGL, Metal, and WebGPU, via ML Drift, our next-generation GPU engine, allowing you to deploy models efficiently across mobile, desktop, and web. On Android, LiteRT optimizes this further by automatically prioritizing OpenCL when available for peak performance, while falling back to OpenGL for broader device coverage.

Empowered by ML Drift, LiteRT GPU has achieved a significant leap in efficiency, delivering substantial performance gains that average 1.4x faster over the legacy TFLite GPU delegate, significantly reducing latency across a broad range of models. See more benchmark results in our previous announcement.

To enable high-performance AI applications, we have also introduced key technical advancements to optimize end-to-end latency, specifically asynchronous execution and zero-copy buffer interoperability. These features significantly reduce unnecessary CPU overhead and boost overall performance, fulfilling the stringent requirements for real-time use cases like background segmentation and speech recognition (ASR). In practice, these optimizations can result in up to 2x faster performance, as demonstrated in our Segmentation sample app. For a closer look at the improvements, see our technical deep dive.

The following examples demonstrate how easily you can leverage GPU acceleration with the new CompiledModel API in C++:

// 1. Create a compiled model targeting GPU in C++. auto compiled_model = CompiledModel::Create(env, "mymodel.tflite", kLiteRtHwAcceleratorGpu); // 2. Create an input TensorBuffer that wraps the OpenGL buffer (i.e. from image pre-processing) with zero-copy. auto input_buffer = TensorBuffer::CreateFromGlBuffer(env, tensor_type, opengl_buffer); std::vector<TensorBuffer> input_buffers{input_buffer}; auto output_buffers = compiled_model.CreateOutputBuffers(); // 3. Execute the model. compiled_model.Run(inputs, outputs); // 4. Access model output, i.e. AHardwareBuffer. auto ahwb = output_buffer[0]->GetAhwb(); C++ Copied See more instructions on LiteRT cross-platform development and GPU acceleration from LiteRT DevSite.

Streamlined NPU integration with peak performance

While CPU and GPU offer broad versatility for AI tasks, the NPU is the key to unlock the smooth, responsive, and high-speed AI experience that modern applications demand. However, fragmentation across hundreds of NPU SoC variants often forces developers to navigate a maze of disparate compilers and runtimes. Furthermore, because traditional ML infrastructure has historically lacked deep integration with specialized NPU SDKs, the result has been complex, ad-hoc deployment workflows that are difficult to manage in production.

LiteRT addresses these challenges by providing a unified, simplified NPU deployment workflow that abstracts away low-level, vendor-specific SDKs and handles fragmentation across numerous SoC variants. We have streamlined this into a simple, three-step process to get your models running with NPU acceleration easily:

AOT Compilation for the target SoCs (optional): Use the LiteRT Python library to pre-comp