Medical Errors and Diagnostic Imaging
Medical error is one of the top preventable causes of death in the United States. Delayed or missed opportunities for diagnoses (MOD) are common in diagnostic imaging, where incidental findings require additional evaluation to complete the assessment for a potential pathology. At Parkland Health, a safety-net public health system, 1.7% of all CT and MRI studies involve such findings, referred to as Delayed Imaging Surveillance.
Large Language Model to Monitor Radiologist Notes
A HIMSS25 session will demonstrate how a large language model can monitor radiologist notes to ensure patients are protected from medical errors and receive their recommended follow-up appointments. The model was developed by Parkland researchers to identify and flag delayed surveillance recommendations from radiologists’ interpretations. The large language model has been integrated into Parkland’s electronic health record, enabling centralized management and navigation of these cases.
Accuracy and Results
Results show 95% accuracy in identifying imaging that requires follow-up based on physician notes and 85% accuracy in determining the appropriate timing for follow-up. "The large language model is outperforming manual review, which can be cumbersome, time-consuming, and more error-prone," said Alex Treacher, PhD, senior data and applied scientist at PCCI, the Parkland Center for Clinical Innovation. "We found the accuracy of 98.1% for the LLM’s detection of follow-ups based on our experiment."
Session Details
The session "Creating a Large Language Model to Catalog Important Radiologist Recommendations" will be held on Wednesday, March 5, from 3:15 to 4:15 p.m. in the Venetian | Level 5 | Palazzo O at HIMSS25 in Las Vegas. Speakers include Treacher, Albert Karam, vice president, Data Strategy and Analytics, PCCI, and Brett Moran, chief health officer at Parkland Health.
Conclusion
By integrating a large language model into their electronic health record, Parkland Health has developed an effective solution to monitor radiologist notes and prevent medical errors. This technology has shown promising results, outperforming manual review and demonstrating high accuracy in identifying imaging that requires follow-up and determining the appropriate timing for follow-up.
FAQs
- What is Delayed Imaging Surveillance?
Delayed Imaging Surveillance refers to the requirement for additional imaging studies to complete the assessment for a potential pathology. - What is the frequency of Delayed Imaging Surveillance at Parkland Health?
1.7% of all CT and MRI studies at Parkland Health involve such findings. - How accurate is the large language model in identifying imaging that requires follow-up?
The large language model is 95% accurate in identifying imaging that requires follow-up based on physician notes.
