Over the past two years, a wave of imaging models has posted accuracy numbers that, on paper, match or beat average radiologist performance on specific tasks — spotting a fracture, flagging a nodule, triaging a chest film for urgency. Vendors like to lead with those numbers. What they mention less is what "accuracy" was measured against.
Most benchmark studies compare a model against a single read, on a curated dataset, with a clear ground truth established after the fact. Real clinical work looks nothing like that. A radiologist reading a chest film at 2am is working from a messy history, an inconsistent machine, and a patient who doesn't fit the training distribution.
Where the models genuinely help
The strongest evidence isn't for replacement — it's for triage and second opinion. Models that flag likely-urgent studies for earlier review have shown real reductions in time-to-diagnosis for conditions like pneumothorax and intracranial hemorrhage. That's a workflow improvement, not a diagnostic one, and it's still valuable.
The question was never "can a model read a film." It's whether a model plus a clinician beats a clinician alone — and on that, the evidence is much thinner than the headlines suggest.
What to watch for
If you're evaluating a tool for your own practice, ask for performance data broken out by the equipment and population you'll actually be using it on — not just its overall benchmark score. A model tuned on one manufacturer's scanner can behave differently on another. That detail rarely makes it into a sales deck, but it's the one that determines whether the tool holds up in your hands.
None of this is an argument against the technology. It's an argument for reading past the top-line number before it changes how you practice.