Medical information shared with chatbots lacks protection. Health data given to AI chatbots from companies outside the healthcare industry does not receive HIPAA privacy protections that apply to doctors and hospitals.

A New York University research team found that AI can be used to restore a patient’s identifying data, and thus circumvent HIPAA. Threats to the privacy of medical data have long existed. Data leaks and hacker invasions have exposed personal, sometimes embarrassing health care data. As AI becomes increasingly integrated into the health care industry, many people wonder whether AI-powered chatbots and automated receptionists used by doctors truly protect patients’ private medical data.

HIPAA applied to medical data gathered by providers, organizations and agencies subject to the law’s regulations are known as HIPAA-covered entities. However, whether HIPAA applies to medical data gathered by AI depends entirely on who is deploying the technology, according to the research team.

Those HIPAA entities include providers such as doctors and psychologists and their clinics or practices. Health plans — whether from health insurance companies, an employer or the government — are also covered. Essentially, HIPAA applies to any individual or entity that comes into contact with or processes protected health information.

Importantly, however, medical information handed over to chatbots used by companies outside the health care industry do not appear to receive the same protections.

De-identifying Data

Regardless of where it’s collected, health data can be altered to remove HIPAA protections. It is common that protected health data is stripped of identifying information, such as a patient’s name, in a process known as de-identification. This de-identified health data can then be sold to anyone from data brokers to pharmaceutical companies. This a very lucrative industry, and has existed for decades. In the case of the pharmaceutical industry, prescription and insurance information can be purchased to in turn target doctors for marketing purposes.

Re-identifying Data

HIPAA and healthcare providers seem comfortable that the patient’s data is adequately protected and that this practice of selling anonymous data is acceptable. Recently, a team of researchers reported on the ease of re-identifying health care information, raising serious questions over HIPAA’s validity in the age of AI.

The New York University research team found re-identifying data to be trivial. The team believes that HIPAA’s protections “are rapidly becoming outdated,” and demonstrated how AI can be used to examine anonymized patient notes to determine an identity. “We believe HIPAA needs urgent updates to offer more robust protections against the sale of this data and we should exercise care when handling clinical notes,” said one of the researchers.

Source: Jiang LY, Liu XC, Cho K, Oermann EK. Paradox of De-identification:
A Critique of HIPAA Safe Harbour in the Age of LLMs.
 arXiv:2602.08997v1 [Computers and Society (cs.CY)]  9 Feb 2026. https://doi.org/10.48550/arXiv.2602.08997

---