HIPAA and LLMs in a mobile app: what you actually have to do

Adding an LLM to a healthcare app is where the "just call the API" instinct gets expensive. HIPAA does not care that the model is new. It cares that protected health information (PHI) is handled by parties who have signed up to handle it, that it is encrypted, that access to it is logged, and that it does not end up somewhere you cannot account for. An LLM is just another party in that chain, and the moment a patient's data flows into a model call, the model provider is inside your compliance boundary.

This is the concrete version of what that means for a mobile build. It is not legal advice, and where it matters below I am explicit about the line between what we have shipped and what we have not.

First, the honest disclosure

We have built a live HIPAA telehealth app. VocalMD (which our founder built from scratch as the sole mobile engineer, before Inseed, under Rehan's agency LaunchX) is a doctor-patient telehealth product that has been in production for over two years, on both stores, with 100+ doctors and 500+ patients onboarded in the first month and use across 200+ hospitals. It pulls real patient records through an EPIC FHIR integration (patients log in with hospital credentials and the app auto-pulls medications, conditions, and lab reports), and it encrypts all sensitive data end to end using the Virgil Security SDK.

Here is what that does and does not prove, stated plainly because HIPAA is exactly the domain where inflated claims are dangerous:

• We have shipped real PHI handling (FHIR record retrieval, end-to-end encryption, in-app clinical communication) in production, at hospital scale, for years.
• VocalMD integrates with Epic via FHIR only. There is no Cerner integration. Do not read one into this.
• We have not shipped a production LLM inside a HIPAA-regulated app. VocalMD predates the LLM features this post is about. What follows is the compliance architecture we would apply, grounded in the PHI-handling work we have actually done, not a certification we hold.
• We are not SOC 2 certified, not HITRUST certified, and we have not passed a payer or hospital security review as Inseed. If a vendor tells you those things casually, ask for the report. We are telling you we do not have them.

With that boundary set, here is the actual work.

The BAA is the gate, and it decides your model provider

Under HIPAA, any vendor that touches PHI on your behalf is a business associate and must sign a Business Associate Agreement (BAA). That single requirement filters your entire model-provider shortlist before latency, price, or quality enters the conversation.

As of 2026, the major LLM providers do offer BAAs, but almost always on their enterprise or dedicated tiers, not the default developer API you sign up for with a credit card. The practical implications:

• The consumer or default API endpoint is off the table for PHI. The endpoint you prototyped against is usually not the one covered by a BAA. You have to move to the enterprise tier, a dedicated deployment, or a cloud provider's hosted-model service (the large cloud vendors will sign a BAA and offer HIPAA-eligible model endpoints).
• A signed BAA changes the data-handling terms. Under a BAA, the provider commits to not training on your data, to defined retention, and to breach notification. Without one, none of that is guaranteed, which is why the default tier cannot touch PHI.
• Get the BAA before you build the feature, not after. The provider you like on quality may not offer a BAA at a price you can carry, and finding that out after you have wired the integration is the expensive way to learn it.

The BAA is not paperwork you handle at the end. It is the constraint that picks your provider.

PHI handling: minimize what flows into the model

The safest PHI is the PHI you never send. Before a single token goes to a model, run the input through this order of operations.

1. Do you need PHI in the prompt at all? A surprising number of "AI health features" can run on de-identified or tokenized input. If you can strip the 18 HIPAA identifiers (name, dates, medical record number, and the rest) and replace them with tokens before the model call, then re-identify locally after, the model provider never receives PHI and the BAA question softens considerably. This is the single highest-impact design move and most teams skip it.

2. If PHI must be in the prompt, the provider must be under BAA. No exceptions. If a de-identification pass is not possible for the feature, then the model endpoint is a business associate and gets treated like one.

3. Send the minimum necessary. HIPAA's minimum-necessary principle applies to the prompt. Do not dump a full patient record into context because it was convenient. Send the fields the feature actually needs.

4. Encrypt in transit and at rest, always. TLS for every call is table stakes. For anything you persist (cached model outputs, conversation history), encrypt at rest. On VocalMD we ran end-to-end encryption on the sensitive data via the Virgil SDK; the principle carries directly to any model-adjacent storage.

What to log (and the trap in logging)

HIPAA requires an audit trail: you need to be able to say who accessed PHI, when, and why. LLM features create a nasty tension here, because the natural thing to log for debugging (the full prompt and the full response) is very often PHI, and now your log store is a PHI store that needs the same encryption, access control, and BAA coverage as everything else.

The pattern that keeps you clean:

• Log the access event, not the content. Record that a model call happened, the user and clinician identity, the timestamp, the feature, the record IDs involved, and a prompt hash. That satisfies the audit trail without turning your logging pipeline into an unmanaged PHI lake.
• If you must log content for quality, encrypt it and scope access. Treat the content log as PHI: encrypted at rest, access-controlled, retention-limited, covered by the same BAAs. Most teams do not want this, which is the point of logging the event and not the content.
• Keep model-provider logging off. Under a proper BAA on an enterprise tier, you can usually disable or limit the provider's own retention of your prompts. Confirm this in writing. Default-tier prompt logging on the provider side is one of the ways PHI leaks out of your control silently.
• Alert on the security-relevant errors. Auth failures and access anomalies on PHI endpoints are audit-trail material, not just ops noise.

What should never leave the device

Some data is better handled where it never becomes a compliance question for a third party at all. If a piece of the feature can run on-device (classification, redaction, entity extraction, a first-pass triage), then that PHI never travels, never hits a model provider, and never needs a BAA to cover it. This is the healthcare case where the on-device vs cloud decision leans hard toward the device, not for latency but for compliance surface.

Concretely, strong candidates for on-device processing in a health app:

• The de-identification / redaction pass itself. Stripping identifiers on the phone before anything goes to the cloud means the raw PHI never leaves. This is the ideal place for it.
• Draft-stage or private-scratch content the user has not yet chosen to share.
• Simple local classification (urgent vs routine, category tagging) that does not need a frontier model.

Whatever genuinely needs a frontier model's capability, and cannot be de-identified, goes to the cloud, to a BAA-covered endpoint, with minimum-necessary input and event-level logging. Everything else stays on the phone.

The checklist

Before an LLM feature ships inside a HIPAA-regulated mobile app:

1. BAA signed with the model provider (enterprise or hosted tier, not the default API).
2. De-identification pass in place, so the model receives the least PHI possible (ideally none).
3. Minimum-necessary prompts: only the fields the feature needs.
4. Encryption in transit (TLS) and at rest (any cached output or history).
5. Event-level audit logging (who, when, what record, prompt hash), with content logging either off or itself treated as PHI.
6. Provider-side prompt retention disabled or contractually limited, confirmed in writing.
7. On-device handling for redaction, drafts, and simple classification, so that PHI never travels.
8. A real security review by someone qualified, before launch, not after. We build to this architecture; we do not substitute for that review.

Where we fit, honestly

We can build the mobile side of this: the FHIR integration, the end-to-end encryption, the de-identification pipeline, the on-device redaction pass, the event-level logging, the BAA-covered cloud integration. We have shipped the PHI-handling half of it in production for years on VocalMD. We have not shipped a production LLM inside a regulated health app, and we are not a compliance certifier. On a healthcare engagement we work alongside your compliance and legal function rather than pretending to replace it.

If you are scoping an AI feature for a health app and want the mobile architecture mapped against this checklist, that is a natural Mobile App Audit, and it is the kind of build we go deeper on for healthcare apps specifically.

---

Shuhel Khan is the founder of Inseed. We integrate AI into existing mobile apps, build AI-native MVPs, and ship mobile development for web-first companies. This post is engineering guidance, not legal advice. Last revised 2026-07-28.