When HIPAA Meets AI: A Health Tech Founder's Guide to BAAs, PHI Training, and OCR Enforcement

Most health tech founders we talk to share a common assumption: HIPAA only applies to hospitals, insurers, and large health systems. It is a reasonable instinct — and it is wrong often enough to create serious legal exposure. The moment your AI-powered health app processes protected health information (PHI) on behalf of a covered entity, or your AI vendor processes PHI on your behalf, the Business Associate Agreement (BAA) requirement kicks in. And once you are a business associate, the HHS Office for Civil Rights (OCR) can investigate you, fine you, and hold you directly liable for HIPAA violations — independent of any covered entity relationship.

This guide walks through the five issues that most commonly trip up health tech startups deploying AI: where AI vendors fit in the HIPAA regulatory chain, when training AI on PHI crosses the line, what your BAAs must say about AI-specific data handling, what OCR enforcement looks like right now, and how to tell whether your product is even subject to HIPAA at all.

The Covered Entity / Business Associate / Subcontractor Chain — Where AI Vendors Fit

HIPAA regulates three categories of entities: covered entities (health plans, health care clearinghouses, and health care providers that transmit health information electronically in connection with covered transactions), business associates, and subcontractors of business associates. The regulatory chain is straightforward in theory and complex in practice.

Under 45 CFR § 160.103, a business associate is a person who, on behalf of a covered entity, "creates, receives, maintains, or transmits protected health information for a function or activity regulated by this subchapter." That definition expressly includes functions like data analysis, data processing, claims processing, quality assurance, and practice management — all activities that AI tools routinely perform. The definition also captures Health Information Organizations, e-prescribing gateways, personal health record providers, and any subcontractor that creates, receives, maintains, or transmits PHI on behalf of a business associate.

For health tech startups, the critical question is whether you are providing services on behalf of a covered entity. If a hospital licenses your AI-powered clinical decision support tool and feeds it patient data so it can generate recommendations, you are a business associate. If an insurer uses your AI claims-processing platform to adjudicate claims, you are a business associate. And if you, in turn, use a cloud infrastructure provider or an LLM API that touches that PHI, those vendors are your subcontractors — and under 45 CFR § 164.502(e), you must flow down equivalent BAA obligations to them.

This cascading liability structure is where startups get caught. A covered entity signs a BAA with you. You sign a BAA with your cloud provider. But you forget to sign a BAA with the AI model vendor that processes PHI through its API — or you sign one that does not address training prohibitions. The gap in the chain is the gap where OCR finds violations.

When AI Training on PHI Becomes an Impermissible Use

One of the most dangerous assumptions in health tech AI is that if a business associate can lawfully receive PHI under a BAA, it can use that PHI to train its models. It cannot — unless the BAA expressly permits it.

Under 45 CFR § 164.502(a)(3), a business associate may use or disclose PHI "only as permitted or required by its business associate contract or other arrangement." The regulation further provides that a business associate "may not use or disclose protected health information in a manner that would violate the requirements of this subpart, if done by the covered entity." Training an AI model on patient data is not a function that covered entities themselves are permitted to perform under the Privacy Rule's default treatment, payment, and health care operations exceptions. Unless the BAA specifically authorizes the use of PHI for model training — and most standard BAAs do not — using PHI to train AI is an impermissible use and a direct HIPAA violation.

This issue is significant enough that we have written a dedicated deep dive on when ML training crosses the BAA line. The short version: if your AI vendor wants to train on PHI, the BAA must explicitly permit it, the use must fall within the scope of services performed on behalf of the covered entity, and the vendor must comply with all Privacy Rule requirements — including minimum necessary limits. In practice, most covered entities will not agree to this, which is why de-identification becomes the safer path.

De-Identification as the Compliance Off-Ramp

HIPAA's de-identification standards, found at 45 CFR § 164.514, offer two methods for rendering health information no longer considered PHI: the Safe Harbor method (removing 18 specified identifiers) and the Expert Determination method (a qualified statistician certifying that re-identification risk is very small). Once data is properly de-identified, it falls outside HIPAA's scope — meaning AI vendors can use it for training without BAA restrictions. But de-identification is not a rubber stamp. Safe Harbor requires removing all 18 identifier categories, and Expert Determination requires documented statistical analysis. Startups that cut corners here are creating enforcement risk, not avoiding it.

What BAAs Must Say About AI-Specific Data Handling

A standard BAA template — even one that technically satisfies 45 CFR § 164.504(e)(2) — is often insufficient when AI is involved. The regulation requires that a BAA establish the permitted and required uses and disclosures of PHI, prohibit uses and disclosures beyond what the contract allows, require appropriate safeguards, mandate breach reporting, and ensure subcontractor compliance. But AI introduces data handling patterns that drafters of standard BAA templates never contemplated.

For health tech startups deploying AI, your BAAs should address at least the following AI-specific provisions:

• Training prohibition: Expressly state whether the business associate may use PHI to train, fine-tune, or improve AI models. If the answer is no — and it usually should be — say so explicitly. Ambiguity in a BAA is not your friend when OCR comes asking.
• De-identification requirements: If de-identified data will be used for model training, specify the method (Safe Harbor or Expert Determination), who performs it, and how compliance will be documented.
• Subcontractor flow-down for AI APIs: If your AI tool calls a third-party LLM or inference API that touches PHI, that vendor is a subcontractor. The BAA must require equivalent restrictions, including training prohibitions, to flow down to every layer.
• Breach notification timelines: The HIPAA Breach Notification Rule (45 CFR §§ 164.400–414) requires business associates to notify covered entities of a breach without unreasonable delay and no later than 60 days after discovery. For AI systems that may exfiltrate PHI through model outputs or inference logs, define what constitutes a "breach" and how quickly the vendor must report it.
• Data return and destruction: 45 CFR § 164.504(e)(2)(ii)(J) requires that at termination, the business associate return or destroy all PHI. For AI vendors, this raises the question of whether PHI embedded in model weights or training data must be "destroyed." Address this explicitly — because regulators will.

We have written more about the negotiation dynamics of these provisions in our guide to negotiating HIPAA BAAs with digital health vendors.

Recent OCR Enforcement Signals on AI and Health Tech

OCR has not yet brought an enforcement action specifically against an AI vendor for training on PHI. But the enforcement landscape is shifting rapidly, and the signals point toward increased scrutiny of business associates — including technology vendors.

In January 2025, OCR announced two enforcement actions against business associates as part of its new "Risk Analysis Initiative." Both cases involved ransomware incidents at technology vendors — Elgon Information Systems (a medical records and billing support company, $80,000 settlement) and VPN Solutions (a data hosting and cloud services provider, $90,000 settlement). In each case, OCR's investigation found that the business associate had failed to conduct an accurate and thorough risk analysis as required by 45 CFR § 164.308(a)(1)(ii)(A). OCR characterized risk analysis as "the foundation for effective cybersecurity and the protection of ePHI."

Simultaneously, OCR published a sweeping Notice of Proposed Rulemaking (NPRM) on January 6, 2025 to modernize the HIPAA Security Rule for the first time in over two decades. The proposed rule would eliminate the distinction between "addressable" and "required" implementation specifications, mandate annual risk analyses, require technology asset inventories and network maps, and — critically for AI vendors — require covered entities to obtain written verification from their business associates at least annually that technical safeguards are deployed. The NPRM also references the emerging threat of AI-assisted attacks as a justification for the overhaul. While the final rule is not yet in effect, it signals OCR's direction clearly: business associates will face more prescriptive, testable, and enforceable security obligations.

According to the HIPAA Journal, more than 700 large healthcare data breaches have been reported each year in recent years, with a large proportion occurring at business associates. OCR has taken note, and its enforcement priorities are following the data.

For health tech startups, the takeaway is direct: even if your AI tool is not the target of an AI-specific enforcement action today, you are subject to the same Security Rule obligations as any other business associate. If OCR investigates a breach at a covered entity and traces the PHI flow to your systems, your risk analysis, safeguards, and BAA compliance will all be examined.

The Wellness App vs. Regulated Health Tool Line

Not every health app is subject to HIPAA. The threshold question is whether your product is a covered entity, a business associate, or neither. If the answer is "neither," HIPAA does not apply — but other privacy regimes do.

Under HHS guidance, when a health app collects information directly from consumers — rather than through or on behalf of a covered entity or business associate — it generally falls outside HIPAA's regulatory framework. Examples include wellness trackers, nutrition logging apps, meditation apps, and sleep monitors that users interact with independently of any health care provider or insurer. As one legal analysis explains, "the key determining factor of whether HIPAA applies to your health app is the source and flow of consumers' health information."

But this line is easy to cross. Common ways a "wellness app" becomes a HIPAA business associate include:

• Integrating with a health care provider's EHR: If your app receives PHI from a hospital or clinic, even through an API, you are likely a business associate.
• Offering your app "on behalf of" a covered entity: If a hospital white-labels or recommends your app and receives patient data through it, the "on behalf of" relationship triggers BAA requirements.
• Processing claims or providing data analysis services to a health plan: Even if your tool feels like a consumer product, if the data flow is from a covered entity, HIPAA applies.

When HIPAA does not apply, other federal and state laws fill the gap. The FTC enforces Section 5 of the FTC Act against unfair or deceptive data practices and administers the Health Breach Notification Rule for non-HIPAA health apps. State privacy laws — including California's CCPA/CPRA, Washington's My Health My Data Act, and others — impose their own restrictions on health data processing. We discuss these overlapping frameworks in our article on mental health app data privacy beyond HIPAA.

The practical risk for startups is not that HIPAA applies when it should not — it is that founders assume it does not apply when it actually does. If your AI-powered health tool touches PHI from a covered entity in any way, you need to treat HIPAA compliance as a foundational requirement, not an afterthought.

Actionable Next Steps

If you are building or deploying AI in a health tech context, here is what we recommend doing now:

1. Map your data flows. Document every path by which PHI enters, passes through, and exits your systems — including AI model APIs, cloud infrastructure, and analytics pipelines. If PHI from a covered entity touches your systems, you are likely a business associate.
2. Execute BAAs with every entity in the chain. Ensure you have a signed BAA with every covered entity you serve and a flow-down BAA with every subcontractor that touches PHI — including AI API providers, cloud hosts, and analytics vendors.
3. Add AI-specific provisions to your BAAs. Explicitly prohibit (or carefully permit) training on PHI, specify de-identification methods if training is allowed, define breach notification obligations for AI-specific scenarios, and address data destruction in model weights.
4. Conduct a compliant risk analysis. OCR's Risk Analysis Initiative makes clear that this is a top enforcement priority. Your risk analysis must be accurate, thorough, and documented — not a gap analysis or checklist exercise.
5. Clarify your regulatory status. If you believe your product is outside HIPAA because it collects data directly from consumers, validate that assumption with legal counsel. The cost of being wrong is an OCR investigation, not just a compliance note.
6. Plan for the proposed Security Rule changes. Even though the NPRM is not yet final, the direction is clear: more prescriptive requirements, annual risk analyses, asset inventories, and written verification from business associates. Start building toward these standards now.