The Most Confident Doctor in America Has Never Seen a Patient

Written by Grace Dahl

Clinical judgment is something only various medical degrees can provide. When we go to doctors and ask for their advice, we’re putting faith in the value of their education and the keen discernment that comes with it. And we know it works—the rise of shows like The Pitt (and, of course, Grey’s Anatomy) reminds us just how much we value those who serve with impeccable bedside manner and unparalleled expertise. Medical competence feels good. So why do we let that all go and place our trust in artificial intelligence (AI) instead?

More than 200 million people—the equivalent of more than half of America—turn to AI every weekwith questions related to health and wellness. However, at best, AI acts as a partially informed doctor without proper education but with overwhelming confidence. AI might speak with authority, but how do we know when its responses are backed by real knowledge or experience? Large language models (LLMs) didn’t learn the fundamentals of nursing through hands-on clinical training, and they didn’t go through years of residency to prepare.

AI has never held a scalpel, a stethoscope, or a thermometer.

AI chatbots are giving flawed medical advice at least 50%of the time, including advice that could be deemed highly problematic 20% of the time. This is a terrifying statistic for any healthcare provider. A real doctor could lose their license if this were true for them. Yet, people are trusting AI to do everything from diagnose their conditions to generate guidance on vaccines and nutrition. And this becomes dangerous quickly as misinformation gets amplified rapidly through these channels.

While we know that the healthcare industry is full of rising costs and barriers for many, the solution can’t be that those locked out of it are forced to rely on the malpractice of AI.

Our health cannot be collateral damage in the AI boom.

People won’t stop turning to AI for answers. So, we need to help build the knowledge base it works from. That requires healthcare organizations being just as invested as providers to ensure that crucial—and correct—data reaches the patient population. A pharmaceutical company needs to be sure that its own product information is superseding any incorrect dosage shared on third-party forums. A health insurance provider needs to be sure enrollment information and access to care is clearly available and not muddled by outdated directories. And a key healthcare resource needs to ensure its critical data is reaching patients, policymakers, and anyone who has hard decisions to make.

We can’t send AI to medical or nursing school, but we can make sure LLMs learn from those who have that education.

For the health communications industry, this means there needs to be an understanding that AI is both a channel and the ultimate consumer of all content produced. This might be a daunting realization, but there are clear next steps to take. Here are a few:

Owned content is one of the best returns on investment. Controlled content is a ripe opportunity for public education. Ensure external-facing content is building long-term authority traceable through searches and AI-generated responses.
Pitching the right outlet matters. Earned media continues to have the greatest reach for AI and beyond, and industry and specialty media often win out over broad legacy, meaning where stories are published can dictate how the information in them is used. Place stories in targeted outlets aligned with smart strategy.
Structure can make a real difference. LLMs reward clarity, so straightforward structure and Q&A formats help ensure accurate health data gets through in AI answers. Prioritize structured, cited content on the facts that an audience needs to know. Include clear headers, clean in-text data, and plain language. Present data in similar language to how your audience will be searching for it.
Know your audience. AI authority shifts by audience, by prompt, and by issue. A patient, provider, or policymaker can ask the same question to an LLM, and all receive a difference answer. Complete an audit to know exactly how your organization and key issues are appearing now in AI responses—the first step to addressing the narrative gap.

This is why we need tools like Signal’s new AI Frame to map out exactly where and how your organization shows up across LLMs like ChatGPT, Claude, Gemini, Perplexity, Grok, and more—so you know what to defend, what to build, and what to correct.

AI Frame will identify gaps or hubs of misinformation and give you a clear plan of action. We have to know where things go wrong because there are real lives at stake and real patients who turn to these platforms for guidance. The least we can do is ensure that they’re met with competence that feels good—not confidence that feels flimsy and knowledge that succumbs to inaccuracy. Let’s make AI smarter and patients safer.