HIPAA and AI — When ML Training Crosses the BAA Line

The BAA Is Not a Blank Check

Healthcare organizations signing agreements with AI vendors often treat the Business Associate Agreement as a formality — a checkbox on the way to deployment. Sign the BAA, get the service, stay compliant. The assumption is that as long as a vendor is willing to sign a BAA, whatever they do with your patients' data inside that relationship is covered.

That assumption is wrong, and it is becoming a significant source of regulatory exposure as AI tools proliferate across clinical and administrative workflows. The HIPAA Privacy Rule does not give business associates broad license to use protected health information for any purpose that benefits their business. It grants them exactly two narrow permissions to use PHI for their own purposes — and neither one encompasses training artificial intelligence models.

This post examines where the line is, why common contract language obscures it, why de-identification is not the escape hatch many assume it is in the ML context, and what compliant AI training actually looks like. If your organization uses AI-powered tools that touch PHI, the questions at the end of this post belong in your next vendor review.

What HIPAA Permits a Business Associate to Do with PHI

The foundational rule under 45 C.F.R. § 164.504(e) is that a business associate may only use or disclose PHI as permitted or required by its contract with the covered entity — and that contract cannot authorize uses that would violate HIPAA if performed by the covered entity itself. The BAA is a constraint, not an authorization mechanism for anything the parties want to permit.

When it comes to a business associate using PHI for its own purposes — rather than simply processing it on the covered entity's behalf — HIPAA grants only two paths under § 164.504(e)(4): the business associate may use PHI (1) for its own proper management and administration, or (2) to carry out its legal responsibilities. That is the complete list. There is no third category for product improvement, no category for aggregate analytics benefiting the vendor's platform, and no category for training generalized AI models.

The "data aggregation" permission that often appears in BAA discussions is separate: it permits a business associate to aggregate PHI across multiple covered entities solely to facilitate healthcare operations for those covered entities. That permission runs to the covered entity's benefit, not the vendor's. A vendor aggregating data across its customer base to build a better model for all customers — or for new customers — is not performing data aggregation in the HIPAA sense.

The practical upshot is stark: an AI vendor that uses PHI from your patient population to train or improve its models — even models that will eventually benefit your patients — is operating outside the two narrow permitted purposes unless it has obtained patient authorization or satisfies HIPAA's research exception with a proper IRB or privacy board waiver. The BAA does not change this analysis. A BAA clause purporting to authorize ML training with PHI does not make that training lawful; it makes both parties potentially liable.

How "Service Improvement" Language Quietly Enables the Violation

Walk through the standard terms of service and data processing addenda from most enterprise AI vendors in healthcare, and you will find language that sounds benign but carries significant legal weight. Phrases like "to improve the service," "for analytics and product development," "to enhance model performance," or "for aggregate insights" appear routinely. These provisions are written by vendor counsel who understand exactly what they mean. Healthcare compliance officers reviewing them may not.

What this language does, operationally, is authorize the vendor to feed patient data into its training pipeline. "Service improvement" in the AI context frequently means retraining or fine-tuning models on real-world usage data — including the PHI that flows through the system when your clinicians use it. "Analytics" can mean not just reporting on your organization's metrics but building statistical models from pooled data across all customers. The outputs of that training are then embedded in the product sold to every subsequent customer, including your competitors.

From a HIPAA standpoint, this use does not fit "proper management and administration" of the business associate, which refers to running the business associate's own operations — legal compliance, HR, billing — not building commercial AI products. It does not fit "data aggregation relating to the health care operations of the covered entity" because the aggregation serves the vendor's commercial interests, not the covered entity's healthcare operations. And it almost certainly does not constitute the kind of "research" that can be authorized through an IRB waiver without significant additional process.

As attorneys Aaron Maguregui and Jennifer Hennessy of Foley & Lardner noted in a June 2025 analysis of AI contracts in healthcare, "many agreements use imprecise language like 'improving services' or 'analytics purposes' to describe what the vendor can actually do with the data. In the health care context, this can be legally problematic." A mismatch between the commercial agreement and the BAA creates compliance risk for both parties — and in audits or breach investigations, OCR will examine both.

De-Identification Does Not Solve the Problem

The standard response from vendors when this issue is raised is some version of: "We de-identify the data before training." This answer deserves careful scrutiny, because de-identification in the machine learning context faces challenges that the HIPAA framework — written before modern ML techniques existed — does not fully address.

HIPAA provides two de-identification methods under 45 C.F.R. § 164.514(b): the Safe Harbor method, which requires removal of 18 specific identifiers including all dates, geographic data below the state level, and any other information that could reasonably identify an individual; and the Expert Determination method, which requires a qualified statistician to determine that the risk of identification is "very small." Once properly de-identified under either standard, data is no longer PHI and HIPAA's restrictions do not apply.

The problem is that machine learning models, particularly large language models and deep learning systems trained on high-dimensional clinical data, can exploit so-called quasi-identifiers — attributes that are not direct identifiers but that, in combination, substantially narrow down who an individual might be. Age, sex, diagnosis codes, procedure codes, geographic region, and admission dates are all individually innocuous. Combined, they can uniquely identify patients in datasets that nominally comply with the Safe Harbor standard.

Research published in the Journal of the American Medical Informatics Association and PMC has repeatedly demonstrated high re-identification precision when combinations of tokenized or hashed demographic attributes are used — even after standard anonymization procedures. A 2024 study in JMIR AI showed that reidentification of participants was achievable from shared clinical data sets that had been processed through conventional de-identification workflows. The Expert Determination method requires documenting that the risk of re-identification is "very small" given the anticipated recipient and their access to other data sources — but ML vendors with access to large external datasets and sophisticated inference capabilities represent exactly the kind of recipient for whom that analysis is most uncertain.

Even setting aside re-identification risk, the Minimum Necessary standard under 45 C.F.R. § 164.502(b) creates an independent problem. HIPAA requires that uses and disclosures of PHI be limited to the minimum necessary to accomplish the permitted purpose. Most AI training pipelines are designed to be data-hungry — models perform better with more data, broader feature sets, and longer time horizons. A vendor that ingests broad patient histories, longitudinal records, and high-dimensional clinical notes to train a general-purpose clinical AI is not collecting the minimum necessary for any discrete permitted purpose. The Minimum Necessary standard and ML training design are in structural tension.

OCR Has Not Issued AI-Specific Guidance — and That Uncertainty Cuts Both Ways

As of mid-2026, the Office for Civil Rights has not issued guidance specifically addressing AI model training with PHI. OCR's December 2024 Notice of Proposed Rulemaking — the first proposed update to the HIPAA Security Rule since 2003 — acknowledged that "businesses are expected to use electronic PHI in accordance with the current HIPAA privacy and security requirements" when deploying AI, but declined to propose specific AI training rules. As McGuireWoods attorneys noted in their January 2025 analysis, "OCR had the opportunity to set new ground rules for the use of artificial intelligence and machine learning in the proposed rule" and instead chose to request additional input.

That regulatory gap is sometimes cited as reason for optimism: if OCR has not said AI training is prohibited, perhaps it is permitted. This reading inverts the structure of HIPAA. The Privacy Rule operates on an opt-in model for uses and disclosures: uses and disclosures not expressly permitted are prohibited. The absence of AI-specific guidance does not create a new permission. It means that existing requirements — the two permitted purposes for BA own-use of PHI, the Minimum Necessary standard, the prohibition on PHI sales without applicable exceptions — apply in full to AI training use cases, without modification.

What the regulatory gap does mean practically is that covered entities and business associates must work through the analysis themselves, without the guardrails of authoritative OCR guidance. Organizations that defer action until OCR issues AI-specific rules are assuming a risk that OCR, applying existing principles to novel facts, will reach conclusions they have not anticipated. The Foley & Lardner guidance published in May 2025 recommends that digital health privacy officers proactively "monitor OCR guidance, FTC actions, and rapidly evolving state privacy laws" — precisely because the direction of regulatory travel is clear even if the final destination is not yet specified.

What Compliant ML Training Actually Looks Like

There are lawful pathways for using PHI to train AI models. Understanding them is as important as understanding the prohibitions.

Complete de-identification. If PHI is fully de-identified under HIPAA's Safe Harbor or Expert Determination standards before training, it is no longer PHI and HIPAA's restrictions do not apply. For organizations with the data infrastructure to perform rigorous de-identification, this is the cleanest path. The constraint is that the de-identification process itself requires access to identifiable data, which must be handled according to the applicable BAA terms, and the outputs must genuinely satisfy the regulatory standard — not merely remove obvious identifiers while leaving re-identification-enabling combinations intact.

Patient authorization. Covered entities may use or disclose PHI for AI training purposes if they obtain valid written authorization from each individual patient under 45 C.F.R. § 164.508. This is operationally demanding at scale, but it is the cleanest authorization route where de-identification is not feasible. Authorization must be specific: a general consent to use data for "research and quality improvement" almost certainly does not satisfy HIPAA's authorization requirements for commercial AI training.

The IRB/privacy board research exception. HIPAA's research exception under 45 C.F.R. § 164.512(i) permits use of PHI without patient authorization if an Institutional Review Board or Privacy Board approves a waiver of authorization. For a waiver to be granted, the IRB or privacy board must find, among other things, that the research involves no more than minimal risk to privacy, that the waiver will not adversely affect the rights and welfare of subjects, and that the research could not practicably be conducted without the waiver. As Davis Wright Tremaine attorneys Adam Greene and Rebecca Williams analyzed in a June 2023 article that remains the most thorough published examination of this issue, AI development may qualify as HIPAA "research" if it is systematic in nature and contributes to generalizable knowledge — including commercial research not intended for academic publication. But that classification does not eliminate process requirements; it shifts the analysis to whether an IRB waiver can be obtained and maintained.

For business associates specifically, the IRB research pathway has an additional layer: the BAA itself must expressly permit the business associate to use PHI for research purposes. As Greene and Williams noted, "this specific research permission is not typical in BAAs and often is a product of negotiation." Organizations considering this pathway should audit their existing BAAs to determine whether they contain — or can be amended to contain — an explicit research permission.

Federated learning and privacy-preserving techniques. Emerging technical approaches such as federated learning (training models on data that never leaves the covered entity's environment) and differential privacy (mathematically bounding re-identification risk in model outputs) offer partial compliance mitigations. These techniques do not eliminate HIPAA's requirements, but they can reduce the scope of PHI processing and provide stronger grounds for Expert Determination de-identification claims. OCR has not issued guidance on these techniques, but healthcare organizations adopting them should document the technical design and de-identification methodology carefully.

Questions to Ask Before Signing Any AI Vendor Agreement

Healthcare organizations evaluating AI vendors should treat the following questions as non-negotiable diligence items before execution. They apply whether you are reviewing a new agreement or auditing an existing one.

• Does the vendor's BAA explicitly enumerate the permitted uses of PHI? A BAA that incorporates the commercial agreement by reference — or that uses open-ended language about "services" — does not satisfy HIPAA's requirement that permitted and required uses be specifically established.
• Does any provision in the BAA or the commercial agreement permit the vendor to use PHI to train, fine-tune, or improve AI models? If yes, examine whether that use fits within the two narrow permitted purposes under § 164.504(e)(4). If it does not, the provision is unlawful and should be struck or replaced with a proper research-authorization mechanism.
• What de-identification methodology does the vendor use, and has it been independently validated? "We anonymize the data" is not an answer. Ask for the methodology — Safe Harbor removal or Expert Determination — and the credentials of the expert if the Expert Determination method is used. Ask specifically whether the analysis accounts for re-identification risk when data is combined with external datasets the vendor has access to.
• Does the vendor train on pooled data across customers? If a vendor uses data from your patient population to improve models that are then deployed to other healthcare customers, that use is not a permitted BA purpose. Ask directly, and require the answer in writing.
• What is the vendor's data retention and destruction policy at contract termination? HIPAA requires that at contract termination, the business associate return or destroy all PHI, or extend BAA protections to PHI that cannot be returned or destroyed. Many AI vendors train on data and embed the statistical properties of that training into model weights that they will not delete. Ask whether model weights trained on your PHI will be retained, and if so, on what legal basis.
• Has the vendor obtained IRB or privacy board approval for any research-classified uses of PHI? If a vendor is characterizing its training activities as HIPAA research, ask to see the IRB protocol and approval documentation. If they cannot produce it, the research classification is not supported.
• Does the vendor's security infrastructure meet the full HIPAA Security Rule requirements? The December 2024 NPRM proposes converting many "addressable" security specifications to "required" — including multifactor authentication, annual compliance attestations to covered entity customers, and 24-hour notification requirements for security incidents. Evaluate vendor security posture against where the Security Rule is heading, not just where it is today.

Key Implications for Practice

The regulatory framework governing AI model training with PHI is not ambiguous — it is simply misunderstood. HIPAA's structure is opt-in for uses and disclosures: permissions must be specific, and the absence of an explicit prohibition does not create an authorization. Business associates have exactly two narrow grounds to use PHI for their own purposes, and generalized AI model training fits neither of them without additional process.

For covered entities, the practical consequence is that every AI vendor relationship involving PHI warrants legal review of both the commercial agreement and the BAA, with specific attention to how the vendor describes its training data practices. Vague "service improvement" language should be treated as a red flag, not boilerplate. Where a vendor's business model depends on pooled training data, that model may not be compatible with HIPAA compliance absent patient authorization or IRB-approved research protocols — which most commercial AI deployments do not have.

For digital health companies building AI-powered products, the compliance architecture should be designed before the technical architecture is locked in. De-identification methodology, data minimization practices, and BAA language governing training rights are easier to get right at the outset than to remediate after a product is in production. Healthcare founders operating across state lines should also track whether state privacy laws impose additional constraints on health data use for AI training — several states with comprehensive privacy statutes treat health data as a sensitive category with heightened consent requirements that may be more demanding than HIPAA's baseline.

The FTC has also signaled awareness of this space. In a February 2024 speech, FTC Chair Lina Khan stated directly that sensitive health data is "simply off limits for model training" — language that does not carry the force of a rule but indicates the direction of federal regulatory concern. OCR enforcement of HIPAA violations in 2024 totaled over $9.9 million across 22 settlements, with BAA deficiencies cited as a contributing factor in numerous cases. AI-related BAA deficiencies are a foreseeable next category.

The bottom line: a BAA is a constraint on what vendors can do with patient data, not a license to do whatever the vendor finds commercially valuable. Healthcare organizations and digital health founders who understand that distinction — and demand that their vendor agreements reflect it — are not just managing compliance risk. They are protecting the patients whose data makes these systems possible.