Skip to content
Physician Case Review Consult

AI-Drafted Patient Messages and Automation Bias

Many clinicians now answer portal messages from AI drafts. When a reassuring draft goes out unchanged, the question is how the reply was reviewed and routed.

Consider a hypothetical. A patient who recently started a blood thinner sends a portal message: a bad headache since yesterday, worse when she bends over. The clinic’s inbox tool drafts a reply suggesting an over-the-counter pain reliever, fluids, and a follow-up visit if it does not improve. A clinician working through a long queue of messages makes a small edit and sends it. The next morning the patient is found to have bleeding in the brain.

Patient portal messages have grown into a large share of clinical work, and AI drafting tools are one of the most common responses. They save time on routine requests. The risk is in the message that only looks routine.

Automation bias. Human factors research has a name for the tendency to accept a machine’s suggestion without the scrutiny the same content would get if a person had written it from scratch: automation bias. It is stronger when people are busy, when the tool is usually right, and when the suggestion is plausible. A well-written draft is persuasive by design. The safety question is not whether clinicians should trust drafting tools, but whether the workflow gives them a fair chance to catch the draft that is wrong.

The routing question comes first. In the hypothetical, the most important failure may have happened before anyone read the draft. A new headache in a patient on an anticoagulant is a symptom many clinics would want triaged by a nurse or escalated the same day. Did the message go into a general queue? Did any rule or tool flag it? Did the drafting tool treat it like a refill request? Routing rules and triage protocols are often the clearest evidence of whether the system was designed to catch messages like this one.

What evidence may exist. None of this appears on the printed chart. Ask for:

  • The original AI draft and the sent message. The difference shows what the clinician changed.
  • Message routing and timing. Which pool the message entered, who opened it, how long it sat, and how long the reply took to send.
  • Triage protocols for portal messages in effect at the time, including any rules for high-risk symptoms or medications.
  • The tool’s configuration, including what patient information it could see (for example, the medication list) and whether drafts were labeled as AI-generated.
  • Governance and training records: approval of the tool, instructions to clinicians on reviewing drafts, and any monitoring of draft quality.
  • Patient-facing disclosure: whether patients were told that replies might be drafted with AI.

What the guidance expects. The Joint Commission and the Coalition for Health AI (CHAI) 2025 guidance on responsible AI use is voluntary. Among its seven elements are training clinicians on each tool’s proper use and limits, telling patients when AI directly affects their care, where appropriate, and monitoring tools on a schedule scaled to risk. CHAI’s risk categorization approach adds a useful lens: the risk of a tool depends on how directly it reaches the patient and how much a person really reviews its output before it does. A patient message goes straight to the patient. The only safeguard between the draft and the patient is the clinician’s review.

Is the tool regulated? Usually the drafting tool is not an FDA-regulated device. FDA’s guidance on clinical decision support software explains that software aimed at clinicians can fall outside device regulation if it meets four criteria, including that the clinician can independently review the basis for what the software suggests. That puts the weight of oversight on the hospital’s own governance and on the reviewing clinician, which is one reason the governance records above matter.

What to expect from the defense. A clinician who sends an AI draft adopts it as the clinician’s own message, and the reply is judged as a clinical communication. That is true whoever wrote the first version. Expect the defense to point to a reasonable triage system, reasonable training, and a draft that was clinically appropriate for the information the patient gave. The plaintiff evidence that answers it is specific: routing that ignored a high-risk combination, review time that was implausibly short, or a tool deployed without guidance on when not to rely on it. None of these facts settles a case alone. Together they show whether the reply reflected a clinical judgment or the path of least resistance through a busy queue.

Sources

  • Joint Commission and Coalition for Health AI. The Responsible Use of AI in Healthcare (RUAIH). 2025. jointcommission.org
  • Coalition for Health AI. Risk Categorization Tool, version 3. chai.org
  • US Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and FDA Staff. fda.gov

The scenario above is hypothetical. Disclosure: Dr. Bansal is Chief Medical Officer of Diagnose Data, Inc., a healthcare technology company. This article does not describe or endorse any product.

This article is educational and does not constitute legal advice. Safety classifications and voluntary guidance describe reporting and improvement frameworks, not a legal standard of care or any outcome. Sources are listed at the end of the article.