Always-On AI: When Inference Must Fit a Coin Cell

Always-on intelligence is not “a small LLM on a phone.” It is inference that can live inside the standby budget of a sensor, a lock, or a wearable—sometimes measured in microjoules per decision.

Phone NPUs and robot VLAs dominate headlines. A quieter race sits underneath: always-on AI for devices that must listen, watch, or feel for years on a battery. That is the TinyML and MCU-class world—keyword spotting, ECG snippets, occupancy, anomaly detection—where low-power AI inference fails if you treat energy as an afterthought.

The first week of November 2026 clustered three vendor moves that illustrate the stack: Ambiq’s heliaAOT 1.0 ahead-of-time compiler (9 November 2026), Nordic’s nRF71 Series SoCs that put Wi-Fi, Bluetooth LE, Thread, and an on-chip NPU on one battery-oriented chip (3 November 2026), and INTERA’s EAGLE Cognit near-memory accelerator IP for TinyML and neuromorphic workloads (26 October 2026). Alongside them, two arXiv preprints (not peer-reviewed) measured where energy actually goes on local boards and why MAC counts mislead on sub-watt accelerators.

This piece stays in that coin-cell regime. It is not a phone NPU shopping guide and not a robot autonomy essay.

What “always-on” really budgets for

Always-on does not mean the radio and the big model run 24/7. It usually means a duty-cycled pipeline:

1. Sense continuously at microwatts to milliwatts (microphone, IMU, PPG, event camera, radar).
2. Detect cheaply with a tiny classifier or spiking front-end.
3. Wake a heavier path only on interesting events.
4. Ship summaries, not raw streams, when connectivity exists.

Miss the budget on step 2 and the device dies in weeks. Overspend on step 3 and “on-device AI” becomes a marketing sticker on a wall-powered gadget. That is why compilers, near-memory IP, and standby-current SoCs matter as much as peak TOPS.

Ambiq heliaAOT 1.0: company claims on Apollo silicon

On 9 November 2026, Ambiq announced heliaAOT 1.0, an ahead-of-time AI compiler for its HELIA ecosystem. According to Ambiq, the tool converts trained models into optimized C code to cut processing time, energy, and memory for on-device inference on memory-constrained Apollo devices.

Ambiq’s published claims (Apollo510 benchmarks versus LiteRT Micro, per Ambiq’s footnotes: matched per-model conditions at 96 MHz; energy as gross MCU-rail energy per inference; model-level, not battery-life, measurements; MLPerf Tiny referring to reference models, not an official MLPerf submission) include:

• Up to 76% lower inference latency and up to 72% lower energy per inference than LiteRT Micro.
• Workload-group table peaks: health monitoring (ECG/PPG/sleep) 76% / 72% / 56% (latency / energy / firmware image); vision 61% / 51% / 29%; voice/audio 52% / 45% / 47%; MLPerf Tiny reference models 36% / 27% / 37%.
• Broader operator support (65 operators plus custom ops), FP16/FP32 paths without integer quantization, and Ambiq’s claim of approximately 4× faster FP32 inference than LiteRT Micro FP32 in its matched-FP32 tests.
• Retargeting across 15 Apollo SoCs with a one-line compiler configuration change; Ambiq says heliaAOT is already in mass-produced products.

Read those as vendor claims, useful for direction and for prompting your own heliaPROFILER-style measurements—not as guaranteed battery-life multipliers.

Nordic nRF71: company claims for battery Wi-Fi + on-device AI

On 3 November 2026, Nordic Semiconductor announced the nRF71 Series, positioning ultra-low-power Wi-Fi, Bluetooth LE, Thread, and on-device AI on a single SoC for devices meant to “run for years on a battery.”

Nordic’s claims in its announcement include:

• Up to 20 times lower standby power usage than current-generation nRF70 Series Wi-Fi products.
• Integrated Axon NPU that runs machine learning models 10× to 17× faster than the 256 MHz Arm Cortex-M33 alone (Nordic’s comparison), enabling keyword spotting, anomaly detection, and image classification without an external AI processor.
• Dual independent radios (Wi-Fi plus 2.4 GHz multiprotocol from the nRF54 generation) that can receive simultaneously so Wi-Fi and Bluetooth LE/Thread need not time-share blindly.
• First parts nRF7120 (dual-band Wi-Fi 6) and nRF7120E (tri-band Wi-Fi 6E + Axon NPU), each with 4 MB on-chip non-volatile memory and 1 MB user-accessible RAM, sampling with selected customers at announcement time.

Again: Nordic’s figures. Standby multipliers are especially sensitive to measurement mode; treat “years on a battery” as a design goal Nordic is selling toward, then verify on your duty cycle.

INTERA EAGLE Cognit: near-memory IP for TinyML and SNNs

On 26 October 2026, INTERA launched EAGLE Cognit, described as ultra-low-power near-memory AI accelerator IP for SoC integration. INTERA’s positioning emphasizes reducing data movement between memory and compute—the classic energy sink in tiny networks.

Per INTERA’s announcement, Cognit offers:

• configurable matrix-vector processing;
• numerical precision from 2 to 32 bits;
• scalable configurations with 4 or 16 processing elements (architectural testing extending to 32);
• native support for sparse computation and spiking neural networks (SNNs), plus conventional CNNs, FCNNs, and MLPs;
• a tooling story spanning TensorFlow / Keras / TensorFlow Lite and neuromorphic paths via NIR- and MLIR-oriented compilation.

Target applications listed by INTERA include neuromorphic vision, wake-word and acoustic events, health wearables and fall detection, occupancy/security sensing, industrial predictive maintenance, and radar perception. No independent joule-per-inference table appears in the launch note—so cite Cognit as an architectural claim (near-memory + SNN-ready IP), not as a measured coin-cell result.

What the preprints add (not peer-reviewed)

Two recent arXiv preprints (October–November 2026)—explicitly not peer-reviewed—help separate marketing from measurement methodology.

Measuring vision-language energy on small local boards (arXiv:2610.20136)

In the preprint “Seeing at the Edge,” Liao, Kondle, Liu, and Gu characterize total-system energy, latency, and thermal behavior for Qwen2-VL-2B (and a text sibling) on a Raspberry Pi 5 with Hailo-10H NPU versus a Jetson Orin Nano Super. As a preprint, treat numbers as author-reported pending peer review. Their highlighted findings include:

• On the Pi, NPU offloading cut VLM energy per generated token by about two-thirds (they report 65.5%, 1959→676 mJ) versus the Pi CPU backend, with lower mean power and no throttle flag in sustained sessions where the CPU path throttled.
• An image adds fixed and per-token energy, but at typical response lengths decode dominates over image ingestion.
• Pi-NPU and Orin-GPU stacks show similar mean per-token energy in their suite, with a length-dependent crossover near ~40 output tokens in their controlled probe.

The paper compares deployment stacks (quantization + runtime + serving), not raw silicon in isolation—exactly the caution always-on designers need when reading any single percentage.

TinyML Hits the Memory Wall (arXiv:2611.03392)

Bonazzi, Capogrosso, Magno and co-authors benchmark ten sub-watt devices across CNN and tiny transformer workloads. As a preprint, not peer-reviewed. Their central claim: the sub-watt accelerators they measured are memory-bound—latency tracks bytes of weights/activations moved, not MAC count—while MCUs without NPUs are not. Consequences they report include:

• MAC count can misrank models on accelerators (for example SqueezeNet with far more MACs finishing faster than MobileNetV3 on an Alif E7 in their table, because of lower memory traffic).
• NPU advantage over a 1 GHz Cortex-M85-class CPU depends on arithmetic intensity (they cite roughly 1.4× on MCUNetV1 to 8.2× on high-intensity SqueezeNet).
• Accelerators in their energy-above-idle set use on the order of 6–34× less energy than the STM32V8 MCU path they measured, with Alif E7 and Sony IMX500 on their power–performance Pareto front.

For product teams, the actionable preprint lesson is: optimize traffic and standby, not brochure MACs, when the constraint is a coin cell.

Design playbook for coin-cell-class AI

1. Separate always-on from sometimes-on

Put wake-word, simple anomaly, or event filters on the path that can legally burn microwatts continuously. Gate vision transformers and multi-second LLM answers behind a wake. The October–November vendor wave is mostly about making that first tier cheaper—not about replacing the cloud for heavy reasoning.

2. Measure energy the way your battery sees it

Prefer energy above idle and full-system rails when comparing boards (as the TinyML preprint argues). Gross MCU-rail joules (Ambiq’s disclosed method) are useful for compiler A/B tests but are not battery life. Nordic’s standby multiplier is a different metric again—do not multiply incompatible percentages.

3. Attack data movement before MAC count

Near-memory IP (INTERA’s pitch), streaming RNN/LSTM/GRU/SVDF support (Ambiq’s always-on feature list), and memory-bound accelerator behavior (preprint) all point the same way: bytes moved per decision dominate. Quantize, fuse, prune activations, and keep weights in the right memory tier.

4. Co-locate radio and inference carefully

Nordic’s nRF71 story is specifically about Wi-Fi-class connectivity without abandoning battery constraints, with on-chip AI so you need no external accelerator. That only helps if your firmware does not keep the Wi-Fi radio in the wrong power state while the NPU is idle.

5. Keep claims labeled

• Ambiq latency/energy percentages → Ambiq claims on Apollo510 vs LiteRT Micro.
• Nordic standby and NPU speedups → Nordic claims.
• INTERA Cognit → vendor IP claims, no joule table in the launch note.
• arXiv energy papers → preprints, cite as preliminary measurement science.

Where this sits relative to bigger on-device AI

Coin-cell inference and phone NPUs share keywords—“on-device,” “NPU,” “offline”—but not budgets. A Copilot+ PC can spend watts for seconds; a Matter lock may have milliwatts averaged over a day. Robot VLAs need closed-loop milliseconds with cameras and torque; a fall-detection wearable needs days of IMU listening before one expensive classify-and-alert. Mixing those constraints in one BOM conversation is how teams oversize silicon, underspecify standby, or ship a demo that only works on USB power.

The October–November 2026 announcements are useful precisely because they stay in the small-budget lane: compilers that emit tighter C for Apollo MCUs, SoCs that refuse to treat Wi-Fi as incompatible with years of battery life, and IP that attacks data movement for TinyML and event-driven nets. Pair them with preprint measurement habits—energy above idle, traffic versus MACs, decode length as a first-class variable—and you get an engineering story, not a slogan.

Bottom line

Always-on AI succeeds when inference fits the same mental model as a good sensor firmware budget: cheap continuous vigilance, rare expensive thought. In late October and early November 2026, Ambiq pushed an AOT compiler with aggressive claimed latency and energy cuts on Apollo-class MCUs; Nordic put claimed multi-protocol radios and an Axon NPU on one battery-minded SoC; INTERA pitched near-memory, SNN-capable IP for SoC builders. Preprints on on-device VLM energy and sub-watt TinyML hardware—again, not peer-reviewed—reinforce that decode length, memory traffic, and idle power decide winners more often than peak compute marketing.

If your product must wake for years on a cell, design for TinyML and low-power AI inference first. Borrow phone-class NPUs and robot VLAs only after the always-on tier is honest about joules.

Related reading

• On-Device AI in 2026: NPUs, Local Models, and What Still Needs the Cloud
• The New On-Device AI Runtime Stack
• Offline AI: When Robots and Vehicles Must Think Without the Cloud

Images: original illustrations generated for AndBeyond.Tech; no logos or real people depicted.