medicine goes AI

the digital doctor is in

prediction:

5 years from now if you tell your friends "i went to a human doctor to diagnose my illness," they will literally ask you "why, could you not find a ouija board?"

medicine is precisely the sort of bounded field with measurable outcomes and complex inputs where human cognition is simply not that good and where AI can excel.

and it’s coming.

fast.

for comparison, median human scores on the USMLE test used here are typically 230-245. 300 is a perfect score so that implies a ~77-82% range. no human has ever hit 300.

it’s quite a comprehensive test that includes decision making and medical inference, not just factual knowledge. you can see more about it and the methodology above HERE.

humans are programmed not to like this.

“human intelligence and creativity” are our narcissistic wubbies, but there is nothing super special about human learning or logic or inevitable about it being best for all tasks.

many crows can solve puzzles faster than human children.

chimps outperform humans on memory tasks.

and the lossy, imperfect, and slow human seat of cognition is, while a marvel in many ways, terrible in others. working memory is small and so it relies on tricks and biases and preconceptions. it’s not designed to be objective or dispassionate, it’s evolved to be fast enough and accurate enough to make choices that are good enough to impart survival benefit.

assessing large amounts of complex data without falling into biases of recency or preference or presumption is not a strong suit.

humans may not like this, but at the risk of triggering a kafka trap, consider that dismissing it because they do not like it is precisely the sort of non-objective preferencing and bias of which the human mind stands accused.

we have, for all of our history, seen ourselves as top of the cognitive food chain.

but not wanting to be knocked from this perch is not the same as still being qualified to stand there.

much of our resistance is reflex arrogance.

people are inevitably going to howl "nooo! we need people to make sure the AI is right!" but this is already, obviously wrong. (source: university of virginia school of medicine)

The study, from UVA Health’s Andrew S. Parsons, MD, MPH and colleagues, enlisted 50 physicians in family medicine, internal medicine and emergency medicine to put Chat GPT Plus to the test. Half were randomly assigned to use Chat GPT Plus to diagnose complex cases, while the other half relied on conventional methods such as medical reference sites (for example, UpToDate©) and Google. The researchers then compared the resulting diagnoses, finding that the accuracy across the two groups was similar.

That said, Chat GPT alone outperformed both groups, suggesting that it still holds promise for improving patient care. Physicians, however, will need more training and experience with the emerging technology to capitalize on its potential, the researchers conclude.

this is med school spin to downplay what was, in reality, a very striking result that must look to them a great deal like a bright asteroid in the sky looked to a t-rex:

the upshot is this:

• doctors alone scored 73.7% on diagnosing patients even when using google etc.

• doctors using GPT scored 76.3%

• but GPT alone scored 92%.

adding a human hurt the results hugely.

it led to 24 errors in 100 instead of 8. triple the misdiagnosis rate is not the kind of outcome one would be wise to dismiss.

1 in 4 misdiagnosed dropped to 1 in 13.

you tell me which product you want when it’s your health on the line…

sorry docs, but in medicine the question is rapidly shifting from "why should we risk AI hallucination?" to "why should we risk it from humans?"

and the answer is going to do wonders for the price of health care.

the fight to come will not take place at the bedside or in the surgical suite.

the humans will not push back with claims of capability or competence. even in surgery, this is already fading and will soon be gone.

they will fight on power.

it’s going to be a guild system protecting prerogative.

the fight to come will be at medical boards and licensing agencies, the FDA and NIH.

“only humans may prescribe medicine.”

“a human must oversee all surgery and procedures.”

it will be a rear guard action and one destined to fail.

the product is simply too different and the offering too impossible to suppress.

they cannot stop us from asking AI about our health, from linking it to our fitbits and apple watches and dietary and athletic output and using it to become healthier and to diagnose our ills.

all they can do is try to gatekeep us from pharma and procedures and that is just not going to hold up well. the improvement in accuracy and drop in cost will be too great to allow a middleman layer of meddling medical meatspace middleware to stand between diagnosis and treatment, adding cost, delay, and reducing quality.

if the US won’t play, i’ll go overseas or get my drugs shipped in.

but i think the US will play because medicare and medicaid are crushing costs for the federal government and this could drive them way, way down.

i suspect it will be difficult for the medical establishment to stand against this tide and against both the american citizen and the american government.

that seems like a loser of a fight to pick.

i’m sure we’re about to see a deluge of anti-AI stories about some medical calamity caused by machine misdiagnosis or mistreatment.

errors are inevitable.

but as you see such, ask always this:

could a human have done better?

are the number of mistakes higher or lower than humans achieved?

the perfect ought not be the enemy of the better.

and as with the tractor and the loom and the printing press, there is no shame in saying “a machine can do this better than a human.”

the shame is in stopping progress by denying it.

“turn your head and meow”