A framework for extraction of clinical information from radiological mammography reports using large language models and retrieval augmented generation.

Authors

Abstract

Extracting structured information from free-text radiology reports is essential for downstream clinical analysis and decision support. In mammography, this challenge is amplified by heterogeneous writing styles, the absence of standardized terminology across institutions, and limited availability of annotated Spanish datasets. Large language models (LLMs) offer a compelling alternative by enabling few-shot generalization without task-specific fine-tuning. We propose a structured Retrieval-Augmented Generation (RAG) framework that leverages annotated mammography reports as in-context demonstrations, enabling joint named entity and relation extraction without task-specific fine-tuning. The framework is evaluated on Spanish BI-RADS-style mammography reports and achieves performance comparable to fine-tuned BETO models in low-resource settings, while significantly reducing training and deployment overhead. In NER, GPT-based few-shot models achieved competitive performance (0.89-0.94 F[Formula: see text]) relative to the fine-tuned BETO baseline (0.97 F[Formula: see text]), despite requiring no task-specific training. In RE, LLMs showed moderate performance (up to 0.78 F[Formula: see text]), remaining below the supervised BETO model (0.99 F[Formula: see text]), reflecting the greater sensitivity of RE to boundary errors and cross-sentence context. Inference costs were low (fractions of a cent per report) and latency remained within seconds, enabling practical deployment scenarios. Local open-weight models preserved privacy and runtime efficiency but exhibited substantially lower accuracy. Few-shot RAG combined with modern LLMs provides a viable, data-efficient alternative for structuring Spanish mammography reports, particularly in low-resource or rapid-deployment settings. While fine-tuned encoders remain preferable for high-accuracy RE, the proposed framework offers a practical balance between performance, operational cost, and accessibility. A remaining limitation is the need for an initial annotated subset to populate the RAG store; future work will explore weak supervision, multimodal extensions, and cross-site generalization.

Tags

Topics