On-device AI
On-device AI keeps inference and data local instead of in the cloud. It is the right tool for a specific set of problems, and the wrong one for others. We are honest about which, and we build the device-side pieces around the app you already have.
Is this the right tool
Running a model on the phone buys privacy, offline capability, and low latency. It costs you model quality, app size, and battery. That trade is worth making for some features and clearly wrong for others.
The honest default in 2026 is hybrid, not fully on-device. Keep the heavy reasoning in the cloud, and push to the device only the parts that genuinely benefit: on-device speech, local pre-processing, and keeping raw sensitive data encrypted on the phone. Anyone selling you a fully local LLM as a drop-in replacement for a cloud model is overselling what fits on a phone today.
How we build it
On-device AI on React Native is a spectrum. At one end, the OS gives you capable models for free; at the other, you are bundling and quantizing your own. Most real products live in between.
The cheapest on-device AI is the one the platform already ships. On Epiphra we used expo-speech-recognition, which taps the native on-device speech engines, so transcription can start instantly without a server round trip. The OS also offers vision and text models you can call locally.
On-device is as much about where data lives as where the model runs. On Afterlight we used end-to-end encryption so raw journal entries stay encrypted on the device, are never stored in the database, and are never exposed to the practitioner. The device does the sensitive handling; only abstracted output ever leaves.
When a task genuinely needs a local model, we integrate a runtime (Core ML, LiteRT, or an on-device LLM runtime) and quantize the model to fit a phone's memory and battery budget. This is real native work, and we scope it honestly against the size and power cost it adds.
The pragmatic architecture routes each step to where it belongs: on-device for speech, pre-processing, and privacy-critical handling, cloud for heavy reasoning. We draw that boundary explicitly so you know exactly what data ever leaves the phone.
Trade-offs and failure modes we design around
What it costs to run
On-device flips the cost model. Cloud AI charges per inference, so cost scales with usage forever. On-device inference has near-zero marginal cost once shipped: the phone does the work, so a million local inferences cost the same as one. For a high-volume, well-scoped task, that can be the difference between a feature that pays for itself and one that bleeds.
The cost moves up front instead. On-device work is heavier engineering (native runtimes, quantization, device testing) and it spends a real budget you do not pay in the cloud: app size and battery. So the question is not "which is cheaper", it is "where do you want to pay", once, in engineering and device budget, or forever, per inference.
We model both paths against your expected usage before building. For a low-volume feature, cloud is usually cheaper all-in; for a high-volume, privacy-critical one, on-device often wins over time. The same cost-model formula makes that comparison concrete instead of a hunch.
Proof
We are precise here, because on-device is an area teams love to overclaim. Here is what is genuinely device-side in what we have shipped, and where the line is.
Inseed client, led by our founder
A live AI reflection app. The genuinely on-device piece is speech: we used expo-speech-recognition, which runs on the native on-device speech engines, so transcription starts without a server round trip. The heavy reasoning and voice generation run in the cloud (Gemini Live), which is the honest hybrid split.
Upwork project, solo lead AI engineer
An AI self-reflection app (in TestFlight) built privacy-first. Our founder built the agentic pipeline and the encryption architecture so raw journal entries stay end-to-end encrypted on the device, are never stored in the database, and are never exposed to the therapist. Only abstracted output leaves the phone. The generative models themselves run in the cloud.
Straight answer on the gap: we have not shipped a fully on-device LLM in production. Both examples above are hybrid: the device does the speech and the privacy-critical data handling, while the heavy model inference runs in the cloud. That reflects what actually fits on a phone in 2026. If your feature needs true local inference, we would prototype it against your device targets in the audit and tell you honestly whether the quality holds, before you commit to a build.
FAQ
Small and quantized ones, yes; a frontier-quality model, not today. On-device LLM runtimes exist and keep improving, but the models that fit a phone's memory and battery are meaningfully weaker than the best cloud models. For most products the honest answer is a hybrid: on-device for speech, pre-processing, and privacy, cloud for the heavy reasoning. We will tell you which of your features can realistically run local.
Only if the design uses it that way. On-device is genuinely more private when raw sensitive data never leaves the phone, which is how we built Afterlight, where entries stay encrypted on-device and are never stored server-side. But a well-designed cloud path with de-identification and a no-training endpoint can also be private. On-device is a strong privacy tool, not a magic label.
Bundling a model adds real download size (tens to hundreds of megabytes depending on the model) and sustained on-device inference drains battery. These are the true costs of going local, and we surface them before committing rather than after. For a small feature the size hit is often the reason to stay in the cloud.
Yes. On-device pieces (speech, local pre-processing, encryption, a bundled model) are added as native modules and a device-side layer beside your current app, not a rewrite. Your existing screens and data stay where they are. That is the same "no rewrite" approach we take to every AI integration.
Where this goes next
These are the app categories where privacy and offline capability matter most, plus our pricing so you can size the work.
30 minutes, free. We will tell you honestly which parts can run on the phone, what it costs in size and battery, and whether the quality holds for your case. Shuhel will be on the call.
Book a 30-minute call