A Real-World Study of Patient-Facing AI in Primary Care

Artificial intelligence tools have emerged as a potential means of extending the capabilities of health care workers, including in primary care settings. Large language models (LLMs) known as chatbots have performed well in simulations. However, little is known about how these systems perform with real patients.

Harvard Medical School researchers at Beth Israel Deaconess Medical Center have conducted what they believe is the first prospective real-world study of a patient-facing conversational AI system in primary care. The work, published Oct. 8 in The Lancet, takes an important step toward evaluating the technology through the same kind of rigorous clinical testing used for other health care interventions.

Get more HMS news

“Despite the theoretical performance of patient-facing AI in experimental settings, it is unclear how conversationally safe these encounters are in a real-world setting,” said corresponding author Adam Rodman, HMS associate professor of medicine at Beth Israel Deaconess and assistant professor of biomedical informatics at HMS. “Before we begin evaluating them as workload interventions, AI tools must first be shown to operate safely and acceptably in real care workflows.”

With this research, he said, the team set out to assess whether this kind of technology could improve both patient and physician experiences.

Testing AMIE

Rodman and colleagues collaborated with researchers at Google to evaluate the Articulate Medical Intelligence Explorer (AMIE), a medical AI chatbot that patients used from home after scheduling an urgent primary care appointment with a Beth Israel Deaconess physician.

Through a secure text-chat interface, the system asked patients about their symptoms, gathered medical histories, provided possible diagnoses for patients to review with their doctor, and generated a summary for the clinician to review prior to the visit.

Conducted between April and November 2025, the study enrolled 114 patients, 98 of whom completed both the AI interaction and their primary care appointment. Every chatbot conversation was monitored in real time by a board-certified internal medicine physician who could intervene if a patient appeared at risk of harm, experienced significant emotional distress, or asked to end the session or if the physician identified a potential safety concern — such as the need to clarify symptoms, provide emergency care instructions, or address concern about a serious possible diagnosis.

Across all 98 completed encounters, none of the conversations required a safety-stop intervention. Supervising physicians identified one hallucination, in which AMIE stated incorrectly that a past surgery date was in the future, and provided additional clinical clarification in five cases.

“This study helps establish the baseline characteristics of such real-world conversations,” said co-first author Peter Brodeur, HMS clinical fellow in medicine at Beth Israel Deaconess. “These baseline characteristics provide a foundation for all future prospective trials aimed at discovering when human-in-the-loop workflows may be appropriate.”

Patient reception

Patients rated the quality of AMIE’s conversations favorably — as did physician evaluators — across most measures. Moreover, patient attitudes toward AI improved after interacting with the chatbot and remained elevated following their clinician appointment.

“One of the most important questions was simply how patients would engage with a conversational AI system in a real clinical setting,” said co-first author Jacob Koshy, HMS instructor in medicine at Beth Israel Deaconess. “By prospectively observing these interactions, we were able to better understand both the opportunities and the limitations of the technology in the context of routine patient care.”

While patients gave high marks to the system’s ability to listen, explain information, and help them feel at ease, they remained concerned around two areas: the confidentiality of information shared with the system and the chatbot’s honesty and trustworthiness. Researchers said those findings underscore the importance of continued work to build patient trust as AI tools move into clinical care.

“Research suggests that patient trust in medical AI systems flows from relationships with their human providers,” said Rodman. “Future research will need to explore which interaction characteristics can build patient trust and how AI interactions can enhance the patient-physician relationship.”

Clinical possibilities

The findings also suggest the technology may have practical value in clinical workflows.

Primary care providers were able to review an AI-generated transcript or summary before seeing their patients for 44 cases. In 75 percent of those cases, physicians said doing so helped them prepare for the visit, and in 57 percent of cases, they reported it may have influenced their clinical approach.

In only one case did a provider report the interaction as somewhat harmful, citing concern that a patient may have experienced anxiety after AMIE included lymphoma among its possible diagnoses.

Rodman and colleagues emphasized that the study was designed to evaluate safety and feasibility rather than to determine whether AI improves health outcomes.

“Our primary care practice — and our hospital — has a long history of rigorously monitoring the safety of the clinical care we provide, and this study serves as an exemplar of how the safety of AI may be assessed prior to clinical implementation,” said co-senior author Marc Cohen, HMS assistant professor of medicine at Beth Israel Deaconess. “We believe the future involves continued involvement of human clinicians in patient care but see a future where the triad of patient, human clinician, and AI help to provide our patients with the very best evidence-based care possible.”

Adapted from a Beth Israel Deaconess news release.