Google、ブラウザ上で WebGPU を経由して .tflite モデルを実行する JavaScript バインディング「LiteRT.js」をリリース

Google released LiteRT.js, a JavaScript binding of LiteRT. LiteRT is Google’s on-device inference library, previously called TensorFlow Lite.

LiteRT.js runs .tflite models directly inside the browser. Because inference stays local, Google cites enhanced user privacy, zero server costs, and ultra-low latency.

What is LiteRT.js?

It is not a new model format. Rather, Google compiled its existing native runtime to WebAssembly and exposed it to JavaScript.

Earlier web AI solutions, including TensorFlow.js, relied on JavaScript-based kernels. Google describes those as less performant. LiteRT.js instead ships the native cross-platform runtime with its optimizations intact.

Consequently, web apps inherit work done elsewhere. Performance upgrades, quantization improvements, and hardware optimizations built for Android, iOS, and desktop arrive on the web too.

How It Works: One Runtime, Three Backends

Under that runtime, LiteRT.js targets three backends:

CPU uses XNNPACK, Google’s optimized CPU library, with multi-thread support and a relaxed SIMD build.

GPU uses ML Drift, Google’s on-device GPU solution, running through WebGPU.

NPU uses the WebNN API, currently experimental in Chrome and Edge.

Two related rules govern dispatch. First, LiteRT.js does not support partial delegation. A graph cannot split across CPU and GPU.

Second, delegation is all-or-nothing per model. If a model cannot be fully delegated to the chosen accelerator, LiteRT falls back to wasm execution. The CPU path has the widest operator coverage.

Performance

Given those backends, Google team reports two distinct results.

Against other web runtimes, LiteRT.js is up to 3x faster across CPU and GPU inference. That figure covers classical computer vision and audio processing models.

Against its own CPU execution, GPU or NPU delivers a 5–60x speedup. That applies to demanding real-time work like object tracking and audio transcription.

Both benchmarks ran in a controlled browser environment on a 2024 MacBook Pro with M4 Apple Silicon. Google notes results vary with local GPU, thermal throttling, and driver optimization. A “10x” figure circulating alongside the launch does not appear in the announcement.

Getting a PyTorch Model In

LiteRT Torch converts PyTorch models to .tflite in a single step.

However, the prerequisites are strict. Your model must be exportable with torch.export.export, meaning TorchDynamo-exportable. It cannot contain Python conditional branches that depend on runtime tensor values. It also cannot have dynamic input or output dimensions, including the batch dimension.

For size, AI Edge Quantizer configures quantization schemes across different model layers. Pretrained .tflite models are also available on Kaggle and the LiteRT Hugging Face Community.

The Minimal Pipeline

Once converted, the runtime code is short. This is the WebGPU path, verified against @litertjs/core v2.5.2:

Copy CodeCopiedUse a different Browser

import {loadLiteRt, loadAndCompile, Tensor} from '@litertjs/core';

// Wasm files live in node_modules/@litertjs/core/wasm/ or on a CDN.

await loadLiteRt('path/to/wasm/directory/');

const model = await loadAndCompile('path/to/model.tflite', {

accelerator: 'webgpu', // 'wasm' | 'webgpu' | 'webnn'

});

const input = new Tensor(new Float32Array(1 * 3 * 224 * 224), [1, 3, 224, 224]);

const results = await model.run(input);

// Accelerator results live off-heap. Move to CPU, then convert.

const cpuTensor = await results[0].moveTo('wasm');

const output = cpuTensor.toTypedArray();

// LiteRT.js uses manual memory management. Delete every tensor.

input.delete();

for (const t of results) t.delete();

cpuTensor.delete();

That last block deserves attention. LiteRT.js does not garbage-collect tensors. Every Tensor must be deleted explicitly, or the app leaks device memory. The snippet in Google’s announcement post omits this step.

WebNN needs one extra flag. LiteRT.js requires JSPI, which bridges synchronous kernel scheduling with asynchronous device polling:

Copy CodeCopiedUse a different Browser

await loadLiteRt('path/to/wasm/', {jspi: true});

const model = await loadAndCompile('model.tflite', {

accelerator: 'webnn',

webNNOptions: {devicePreference: 'npu'}, // 'cpu' | 'gpu' | 'npu'

});

Before writing pre-processing, test with fake inputs. Run npm i @litertjs/model-tester, then npx model-tester. It runs your model on WebNN, WebGPU, and CPU using random inputs. Use model.getInputDetails() to read input names and shapes.

Use Cases With Examples

Those APIs back four demos Google shipped at launch:

Use caseExampleWhat LiteRT.js provides

Real-time object detectionUltralytics YOLO26, via official LiteRT export in the Ultralytics Python packageOne export path to mobile, edge, and browser

Depth from a webcamDepth-Anything-V2, mapping video pixels into a live 3D point cloudWebGPU execution at interactive rates

Image upscalingReal-ESRGAN, upscaling 128×128 patches to 512×512, reassembled into a 4x imageLocal processing, no upload

Semantic searchEmbeddingGemma vector search running in-pageEmbeddings computed client-side

LiteRT.js vs TensorFlow.js

Those demos raise an obvious question for existing web ML teams.

DimensionLiteRT.jsTensorFlow.js

KernelsNative runtime compiled to WebAssemblyJavaScript-based kernels

Model format.tfliteTF.js graph and layers models

CPU pathXNNPACK, multi-thread, relaxed SIMDJS and Wasm backends

GPU pathML Drift over WebGPUWebGL and WebGPU backends

NPU pathWebNN, experimentalNone

MemoryManual; call .delete()Automatic, plus tf.tidy and tf.dispose

Cross-platform reuseSame artifact as Android, iOS, desktopWeb-only

Importantly, the two are not mutually exclusive. Google positions LiteRT.js as a replacement for TF.js Graph Models specifically, not the whole library.

TensorFlow.js remains the recommended tool for pre- and post-processing. The @litertjs/tfjs-interop package passes tensors between them via runWithTfjsTensors. Avoid tensor.dataSync, which carries a significant penalty on the WebGPU backend.

Interactive Explainer

The embed below animates the six pipeline stages across each backend.

(function(){

var f=document.getElementById("mtp-litertjs-frame");

window.addEventListener("message",function(e){

if(e&&e.data&&typeof e.data.mtpLiteRtHeight==="number"){ f.style.height=e.data.mtpLiteRtHeight+"px"; }

});

})();

Key Takeaways

LiteRT.js runs .tflite models in-browser using Google’s native runtime compiled to WebAssembly.

Reported gains: up to 3x over other web runtimes; 5–60x for GPU/NPU over its own CPU path.

Three backends: XNNPACK on CPU, ML Drift over WebGPU, WebNN for NPUs. No partial delegation; falls back to wasm.

Tensors are manually managed. Call .delete() or leak device memory.

WebNN remains experimental. WebGPU is the practical acceleration target today.

Sources

LiteRT.js, Google’s high performance Web AI Inference — developers.googleblog.com

Get started with LiteRT.js — developers.google.com

LiteRT Torch (ai-edge-torch) — GitHub

AI Edge Quantizer — GitHub

LiteRT-LM — GitHub

The post Google Releases LiteRT.js: A JavaScript Binding of LiteRT That Runs .tflite Models in Browsers via WebGPU appeared first on MarkTechPost.