Latest multimodal large language models show limitations on image-based radiology exam questions.
Key Details
- 1Researchers tested ChatGPT-4v and ChatGPT-4o on 222 image-based multiple-choice questions from national radiology board exams (2020 and 2024).
- 2These LLMs have been recently trained to process both text and images.
- 3Despite advancements, significant concerns remain regarding their reliability for diagnostic tasks in radiology.
- 4The potential of such models in radiology workflows, such as report generation and diagnostic support, is still under early investigation.
Why It Matters
As large language models gain capability for image analysis, assessing their reliability is crucial for safe deployment in radiology. Failures on board-style questions highlight the need for ongoing scrutiny before clinical trust is warranted.
Source
Radiology Business
Related News
•Radiology Business
Real-World Study: Radiology AI Best in Emergency and Inpatient Settings
A commercial AI tool for intracranial aneurysm detection outperformed in inpatient and emergency settings but yielded limited benefits for outpatients in a major health system study.
•Radiology Business
New Rubric Enhances Safety of AI-Generated Radiology Summaries
Researchers developed a five-factor rubric to assess the safety and quality of AI-generated, patient-friendly radiology report summaries.
•Radiology Business
AI Model Surpasses Radiologists in Detecting Subtle Hip Fractures
A new AI model outperformed radiologists in identifying difficult-to-detect femoral neck fractures on radiographs.