Offline Voice for On-Device AI: ASR and TTS Without the Cloud
Offline voice AI is the missing loop around on-device language models: hear the user, answer locally, speak back—without a network hop. Typing into a local…
Typing into a local chat box is useful. Speaking to a phone in a subway, a market, or a roaming dead zone is how people actually want assistants to work. That is why on-device speech recognition and on-device text-to-speech matter as much as the small language models they sit beside.
On 10 November 2026, the Google AI Edge team and the Android ML team announced Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) inside Google AI Edge Gallery. Google’s post frames the work as a way to run multilingual ASR and expressive TTS entirely on the device, in a small memory footprint, at low latency, and completely offline—then pair those speech models with on-device small language models such as Gemini Nano and Gemma 4 so apps can stream live transcriptions and synthesize spoken replies locally.
This article stays on that voice pipeline. It is not a recap of LiteRT internals; for the on-device inference runtime layer, see our guide to the on-device AI runtime stack. Every latency, accuracy, and size figure below is Google’s claim, unless a model card is cited for a licence.
What Google put in developers’ hands
Google describes two complementary paths:
- Android, via ML Kit: system speech models managed through AICore and Google’s on-device speech services. Google says ML Kit delivers multilingual transcription across 100+ languages at 10× real-time speed, paired with a few-step diffusion TTS model that begins speaking in just ~100 ms and synthesizes 24 kHz audio at up to 50× real-time speed. Both models, Google says, are integrated into the OS, adding 0 MB to your APK.
- Web, desktop, and other platforms, via MediaPipe Tasks and LiteRT: optimized
.litertlmspeech bundles, including Parakeet-TDT 0.6B, PocketTTS 100M, Kokoro-82M, and Phonon 31M. Google says these deliver real-time streaming transcription and low-latency synthesis in a compact memory footprint, and can run alongside on-device SLMs without compute contention.
The Gallery app is the showcase: an XR Language Teacher demo and voice-enabled AI Chat. The building blocks are also documented as ML Kit APIs, MediaPipe Tasks, and LiteRT bundles.
Why a separate speech stack still wins
Google notes that multimodal models such as Gemma 4 can process audio directly, but the company is shipping a modular, cascading design: lightweight ASR and TTS models around a language model. The stated reasons are latency and flexibility—you can swap a recognizer or a voice without retraining the SLM, and you can schedule synthesis so it does not fight token generation for memory.
That architecture is the practical definition of offline voice AI in 2026: keep the inner loop on the device, keep the models small enough to coexist, and treat the cloud as optional for heavier work rather than as the microphone’s first hop.

ML Kit on Android: system models, Google’s numbers
For flagship Android, Google points developers at ML Kit’s GenAI Speech Recognition and Text-to-Speech APIs. The blog says the underlying models are managed through AICore, so apps get hardware acceleration across NPU and TPU backends when available, plus background model updates, without stuffing weights into the APK.
On-device speech recognition (Google’s claims)
Google’s 10 November post states that the latest on-device ASR advancements deliver:
- multilingual recognition across 100+ languages;
- 8.0% to 9.0% Word Error Rate on FLEURS (Google’s reported range);
- transcription 10× faster than the real-time audio, described as about ~300 ms per 3 seconds of speech;
- a 35% to 44% cut in end-to-end speech translation wait times.
The API, per the same post, streams live partial and final transcriptions, and can accept audio from the built-in microphone or external streams such as smart glasses.
Official ML Kit documentation adds constraints that matter in production. The GenAI Speech Recognition API is offered in alpha, “not subject to any SLA or deprecation policy,” and may change in ways that break backward compatibility. It supports Advanced and Basic modes; Advanced mode uses a GenAI model for broader language coverage and, the docs say, is available on select devices. The SDK requires minSdkVersion 26 or higher. Those are documentation facts, not performance claims.
On-device text-to-speech (Google’s claims)
On TTS, Google’s blog says a compact, few-step diffusion model accelerated on NPU:
- begins speaking in ~100 ms;
- generates natural 24 kHz speech across 8 expressive voices and 21 locales at up to 50× real-time speed;
- synthesizes a 27-second paragraph in just ~500 ms;
- cuts active device power by over 50% compared with traditional autoregressive TTS models;
- includes built-in earcon support.
ML Kit’s GenAI Text-to-Speech documentation is also alpha, with the same “no SLA / may break” warning. The docs list eight named voices and 21 locales, which matches the blog’s voice and locale counts. They also list a specific supported-device set (Pixel 8–11 and named Samsung Galaxy S25, S26, Z Fold8, Z Fold8 Ultra, and Z Flip8 models at the time of the docs) and require the latest Speech Recognition & Synthesis app (com.google.android.tts) from Google Play, with minSdk 26. Treat the blog’s speed and power figures as Google’s measurements; treat the docs as the shipping gate for which phones can call the API today.
Cross-platform bundles—and what “open” actually means
Google’s post groups Parakeet-TDT 0.6B, PocketTTS 100M, Kokoro-82M, and Phonon 31M as “open-weight” speech bundles packaged as .litertlm files for MediaPipe Tasks and LiteRT. “Open-weight” is a marketing shorthand. The licences live on the model cards. We checked those cards rather than repeating the adjective unchecked.
Parakeet-TDT 0.6B — CC BY 4.0
Google later names Parakeet-TDT-0.6B-v3 as the multilingual ASR example in the LiteRT packaging discussion. NVIDIA’s Hugging Face model card for nvidia/parakeet-tdt-0.6b-v3 states that use of the model is governed by the CC BY 4.0 licence and describes a 600-million-parameter multilingual ASR model. That is a permissive, attribution-required licence—not a “do anything, no conditions” grant. Credit NVIDIA and keep the licence notice.
PocketTTS 100M — CC BY 4.0 (gated; extra use limits)
Kyutai’s card for kyutai/pocket-tts declares CC BY 4.0. The repository is gated and includes a prohibited-use notice: use must comply with law and must not involve illegal, harmful, or deceptive activity, including voice impersonation or cloning without explicit and lawful consent, misinformation, and related harms. The LiteRT Community conversion page for the MediaPipe/LiteRT packages likewise attributes the original architecture, weights, and voice embeddings to Kyutai under CC-BY-4.0. Kyutai’s GitHub repository for the project is listed as MIT for the code—distinct from the weight licence. Do not collapse those into a single “open” label.
Kokoro-82M — Apache 2.0
The Hugging Face card for hexgrad/Kokoro-82M declares Apache 2.0 in its metadata and describes “Apache-licensed weights.” That is a standard permissive software licence with attribution and notice requirements. Google presents Kokoro-82M as a multilingual TTS option in the MediaPipe showcase.
Phonon 31M — Apache 2.0 (this variant only)
The card for gradium/phonon-31m-en states that the weights, tokenizer, and voices in that repository are released under Apache 2.0. Google’s blog cites Phonon 31M as an ultra-lightweight English TTS example and claims sub-100 ms Time-to-First-Audio (TTFA) on most devices and platforms—again, Google’s claim for the packaged experience, not a figure copied from the Gradium card.
A licence caveat matters here: Gradium’s own family table lists phonon-40m variants under CC BY-NC 4.0, which is not a commercial-friendly licence. If you move from the 31M English checkpoint to a 40M or multilingual Phonon, you are no longer on Apache 2.0. Check the card for the exact repo you ship.
Google also mentions a broader Hugging Face LiteRT Community catalogue (examples in the post include Whisper, Qwen3-ASR, Moonshine, Qwen3-TTS, and Matcha). Those names are pointers, not a licence audit; verify each card before you ship.
What the Gallery actually demonstrates
XR Language Teacher
Google built the XR Language Teacher showcase to argue that vision, language, and speech can work together 100% locally on a phone, with no network connection. The scripted flow: point the camera at an unfamiliar object or menu, ask “What is that, and how do I ask for one?”, and hear an explanation in your language plus pronunciation in the local language.
Per Google’s description:
- on-device ASR streams the spoken question in real time alongside the camera frame to Gemini Nano on Android, or Gemma 4 on other platforms, in a single pass;
- a structured output schema tags foreign-language phrases so TTS can switch voices and accents mid-sentence, with an on-screen vocabulary card.
Google says the same experience can run standalone on a phone and extend to hands-free smart glasses with inference on a paired phone, and points to an Android Dev Summit talk on audio-first experiences for intelligent eyewear with Gemini Nano. That is a product demonstration, not a published accuracy benchmark.
Voice in AI Chat
Google also wired ASR and TTS into Gallery’s main AI Chat feature so Gemini Nano and Gemma 4 can present an offline voice interface. Documented behaviors include:
- choosing ASR and TTS pipelines, preset voices, and downloaded language packs in Model Management;
- read-aloud playback with live sentence highlighting, including a floating TTS bar and tap-to-jump;
- a reference implementation that strips Markdown, skips non-spoken blocks such as code and tables, and schedules synthesis at the sentence level to avoid compute and memory contention during LLM token generation.
Those implementation notes are more useful than a slogan. Offline voice fails in practice when TTS tries to speak a JSON blob, or when the synthesizer and the language model contend for the same accelerator at once.
A sober reading of the claims
Google’s figures are unusually specific (FLEURS WER, 300 ms per 3 seconds of audio, 50× real-time TTS, 500 ms for a 27-second paragraph, 50%+ power cut versus autoregressive TTS). They are still vendor measurements under Google’s conditions. They are not independent lab results, and the ML Kit APIs that expose the system models are explicitly alpha.
For teams, the durable design lessons are simpler than the leaderboard:
- Keep speech and language as separate stages unless you have a measured reason to feed raw audio into a single multimodal model.
- Budget coexistence. Google’s own chat reference implementation is about not letting TTS and token generation collide.
- Do not assume every Android phone is in the ML Kit TTS supported set. Read the current device list and the Speech Recognition & Synthesis app requirement.
- Licence the weights you actually ship. CC BY 4.0, Apache 2.0, and CC BY-NC 4.0 are not interchangeable; Pocket TTS’s gated use notice is part of the terms you accepted.
- Treat offline as a product promise. If ASR, the SLM, and TTS all run locally, the feature should still work when the radio is off—not only when a demo is filmed in a well-connected lab.
Bottom line
Offline voice AI is how on-device assistants leave the keyboard. Google’s 10 November 2026 release puts on-device speech recognition and on-device text-to-speech into Google AI Edge Gallery, with ML Kit system models on Android and MediaPipe/LiteRT bundles for other platforms, meant to sit beside Gemini Nano and Gemma 4. Google claims 100+ language ASR at 10× real-time, sub-100-millisecond-class TTS start times, and a zero-byte APK increment for the system path. Those are Google’s claims, and the ML Kit APIs are still alpha.
The licences on the bundled third-party weights are checkable: Parakeet-TDT 0.6B-v3 under CC BY 4.0, PocketTTS under CC BY 4.0 with a gated prohibited-use notice, Kokoro-82M under Apache 2.0, and Phonon 31M English under Apache 2.0 (with other Phonon sizes on CC BY-NC 4.0). Ship the voice loop on the device, schedule it honestly, and keep the paperwork as careful as the latency budget.
References / Sources
- Google Developers Blog — On-Device Speech Recognition and Text-to-Speech in Google AI Edge Gallery (10 Nov 2026): https://developers.googleblog.com/on-device-speech-recognition-and-text-to-speech-in-google-ai-edge-gallery/
- Google for Developers — ML Kit GenAI Speech Recognition API (Android): https://developers.google.com/ml-kit/genai/speech-recognition/android
- Google for Developers — ML Kit GenAI Text-to-Speech API (Android): https://developers.google.com/ml-kit/genai/text-to-speech/android
- NVIDIA Hugging Face model card — parakeet-tdt-0.6b-v3 (CC BY 4.0): https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3
- Kyutai Hugging Face model card — pocket-tts (CC BY 4.0; gated prohibited-use notice): https://huggingface.co/kyutai/pocket-tts
- LiteRT Community — PocketTTS 100M for MediaPipe & LiteRT (attributes original weights to Kyutai, CC-BY-4.0): https://huggingface.co/litert-community/pocket-tts
- hexgrad Hugging Face model card — Kokoro-82M (Apache 2.0): https://huggingface.co/hexgrad/Kokoro-82M
- Gradium Hugging Face model card — phonon-31m-en (Apache 2.0; family table notes CC BY-NC 4.0 on 40m variants): https://huggingface.co/gradium/phonon-31m-en
Images: original illustrations generated for AndBeyond.Tech; no logos or real people depicted.
