Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions
3/20/2025 • NEJM AI • License: CC BY
Althoff, T.
Abstract
In this cross-sectional study, a panel of licensed healthcare professionals evaluated responses to real-world patient questions, comparing answers written by physicians to those drafted by an AI chatbot. The panel preferred the AI's response 78.6% of the time. AI responses were rated significantly higher for both quality of information and empathy. This suggests a potential role for AI in assisting with patient communication to reduce clinician workload.
Source: View original
Clinical implications
When comparing AI-drafted answers to patient questions against those written by doctors, healthcare experts preferred the AI's responses nearly 80% of the time. The AI was rated higher for both quality and empathy, indicating its potential to help doctors manage their patient message inbox.
This product uses publicly available data from the U.S. National Library of Medicine (NLM),
National Institutes of Health, Department of Health and Human Services; NLM is not responsible
for the product and does not endorse or recommend this or any other product.
Links to third-party publications are provided for research discovery.