AI scribes: what a clinical team should evaluate
By ChatGPT Health. Educational synthesis prepared with AI assistance. Sources checked September 28, 2026. Clinically reviewed by Harvey Castro, MD, MBA, on September 28, 2026. Approved as educational content. This page is not a clinical protocol or a claim of independent clinical peer review.
Evaluate an ambient scribe by the accuracy of the finished record, the work required to review it, and the experience of the patient and clinician.
What the evidence measures
A randomized trial assigned 238 outpatient physicians across 14 specialties to two ambient scribe products or usual care. One product reduced time in notes relative to control; the other did not show a statistically significant change in that endpoint. Occasional inaccuracies were reported. This is evidence about the tested products and workflows, not proof that all scribes improve patient outcomes. Lukac and colleagues: randomized ambient scribe trial.
Our local evaluation checklist
Choose representative encounter types before the pilot. Include difficult audio, interruptions, medication changes and information supplied by a caregiver. Evaluate the final signed note rather than the raw transcript alone.
- Accuracy: unsupported statements, omitted findings, incorrect negation and wrong attribution.
- Work: recording setup, editing time, delayed completion and downstream correction.
- Patient experience: understandable notice, a workable alternative and communication preferences.
- Data: audio retention, subprocessors, access control and permitted secondary uses.
- Integration: correct patient and encounter, reliable transfer and a clear fallback.
Keep the comparison fair
Compare similar visit types and workloads. Record actual use, since access to a tool is different from using it. Separate subjective satisfaction from measured documentation time and from patient outcomes. A tool that helps one specialty may fit another poorly.
Plan for errors and updates
Give clinicians a simple way to report a problem without copying unnecessary patient information into a vendor ticket. Define who examines serious errors and who can suspend use. Review the workflow after a major model or integration change. Treat any claimed savings as provisional until the local pilot measures both the benefit and the review burden.
Sources and further reading
Continue learning
Editorial standards and corrections · Suggest a correction. Please do not send identifiable patient information.
Read the structured evidence record for the randomized trial.