An important first step. I am early in my practice and I fully expect medicine as I know it today to become unrecognizable within my lifetime. I am optimistic that it will be done carefully and will ultimately tremendously benefit our patients.
That being said, I had to laugh, this is one hyperbolic headline. There are of course some caveats.
"Few efforts to harness LLMs for medicine have explored whether the systems can emulate a physician’s ability to take a person’s medical history and use it to arrive at a diagnosis. Medical students spend a lot of time training to do just that, says Rodman. “It’s one of the most important and difficult skills to inculcate in physicians.”
This does take a lot of training and experience. The challenge in diagnosis isn't about asking a history and integrating the information, it's about effectively encouraging patients to provide necessary information that they might not realize is relevant or know how to articulate. Different patients often require wildly different approaches. Medical literacy does play a role, but a patient that would say, "Hi doctor, I experienced central chest pain accompanied by discomfort in the upper stomach that happened two hours ago" (from pg 33 of the preprint) is not realistic. More likely you get a vague complaint of "heartburn that started a while ago".
Similarly: "Currently, I'm not on any prescribed medications". More frequently you get something like "I take a blue one" or "Golly Telly" (Go-Lytely) or "Gabatini" (Gabapentin). I do think an LLM could probably parse these but such idiosyncrasies in a history do compound. And though someone may be prescribed a med and think they take it as prescribed, sometimes it takes a hunch and clinical experience to tease out that in fact the med is not being taken at all as indicated.
Moreover, the better bedside manner was assessed via text conversation. I wouldn't quite call that "bedside" manner. I also wonder how such a system will deal with patients that have self-diagnosed themselves and looked up the "right answers" to get what they want -- a difficult reality that takes some parsing to figure out what's real and what's not.
Overall though, the preprint, in contrast to the Nature news article/headline, does a better job of discussing these limitations. I congratulate them on excellent work. Thank god LLMs can't do surgery, though I'm sure my time will come as well.
Medicine as a career offers immense personal fulfillment, variety, human interaction, and prestige at the expense of dealing with difficult outcomes and ranges of personal sacrifice -- neurosurgery as a specialty just takes all of these to their extremes.
I value the former and find ways to discount the latter. So I am very happy. Though sane or not would be up to others.
I'm a physician and use chatGPT extensively for coding, writing, and general knowledge inquiry.
With 60-70% correct rates on most training sets and 0.63 critical errors per report, for any physician not very well-versed with the limitations of LLMs, this is more of a liability than an asset. Some of the biggest barriers to care are cognitive, such as anchoring or availability biases. LLMs in their current state will only muddy the water.
Good physicians already know and do use these tools, bad ones will only get worse. A legal mandate will not benefit care.
Doubtless these models will progress to where this calculus will change. The only benefit from a mandate now that I can foresee is to accelerate fine-tuning by forcing widespread reinforcement learning by physicians, but that is a different discussion.
The way it usually works is that we have anesthesia put the patient entirely to sleep to start and then we wake them for the sensitive part of the case when we need to perform stimulation and cognitive testing. We use local anesthetic like lidocaine and marcaine on the scalp while opening to minimize pain when temporarily awake. Then the patient goes back to sleep for closing. It’s a tough balance for the anesthesiologists to maintain —- they are as critical to the procedure as the neurosurgeon is.
The couple of times I've woken up from anaesthesia, I was apparently pretty rambly and nonsensical for awhile. I wonder if I'd be able to play piano like that. To some extent, playing is automatic, sub-verbal for me. Maybe it would come out great and I just wouldn't remember it, but... how do you gauge the performance given that they're still coming out of anaesthesia?
While I'm asking, there's a question that's been driving me crazy since a recent colonoscopy where I was sedated with fentanyl and (? something like diazepam). I think I remember being awake and remembering most of the procedure immediately afterwards, but within about an hour I couldn't remember anything except one moment when I was in pain and shouting that I was going to explode, as a nurse put another ampule in the drip line. What's bugging me is, was I really experiencing pain like that and aware the whole time, and the drugs just erased my memory of it afterwards? Or was that just a breakthrough moment in an otherwise uneventful procedure where I felt relatively little?
Propofol would be for heavy sedation and would require an anesthesiologist or nurse anesthetist to be present. My insurance didn't cover that. I checked, and what I got was Versed (midazolam) + fentanyl.
At least in my prior experience with benzodiazepines like Xanax or lorazepam, they didn't cause any amnesia. I've never had fentanyl or any synthetic opioid besides during this procedure, so maybe that was what caused the amnesia. Or maybe there was no amnesia, and I was just so stoned from the combination that I was asleep and didn't really experience much except when some pain woke me up... I guess that's what I'm wondering.
Without further information regarding the exact location of the patient’s tumor, it is hard to comment in detail. However, one way handedness affects awake glioma resection is that while right handed people typically have their language dominance localize to the left hemisphere, left handed individuals are more likely to have right sided or even codominant language function. Assuming this was a right sided tumor, the surgeon probably meant that he also had to account for avoiding speech deficits in this left handed patient that he wouldn’t worry about in a right handed one.
The difficulty of a task transcends mere mechanics.
A common analogy used in neurosurgery training: "Imagine a wooden plank on the driveway and walk its length -- no problem. Suspend that same board ten stories in the air and try again." The hard part is not the "figuring out" or "execution" but knowing the irreversibility and making the correct decision. Your patient expects you to be correct 10 times out of 10, yet you know that's not possible. Squaring our fallibility with the irreversibility of our missteps is the hard part, and what keeps surgeons up at night. The fly struggles in the web.
All too common. Despite our best efforts, even the most detailed neuropsychological battery does a very poor job at detecting certain categories of cognitive changes -- especially in patients who were above average at baseline.
I'm a neurosurgeon and we commonly advise our patients: Even after a "perfect" or minimal surgery without any evidence of periprocedural stroke or complication, you may not ever be the same again. Sometimes it takes months for these changes to be noted, sometimes it's only even noticed by family members. Odd word finding difficulties, perception changes, memory/concentration issues; the gamut is endless. As we say, no one's the same when the air hits your brain.
The neurosurgeon meticulously plans the exact trajectories prior to surgery and we use a combination of software and hardware (Stealth or Brainlab) to ensure we follow the planned path. Of course, complications still occur but are fairly rare, serious or permanent ones even more so.
The brain matter is indeed pierced. While “you only use 10% of your brain” is a myth, much of the brain is what we call non-eloquent, meaning it can be damaged or removed without any noticeable effects. Especially when it just means being pierced by a 2-3mm diameter lead. Are there hidden side effects that we are simply unable to recognize? Sure, but with as primitive an understanding of the brain as we currently have, we are forced to rely on a more pragmatic rather than philosophical approach.
