Lesion-aware visio-linguistic context regularization for radiology report generation.
Authors
Affiliations (6)
Affiliations (6)
- Hangzhou Medical College, Hangzhou, 311300, China; Japan Advanced Institute of Science and Technology, Nomi, 923-1292, Japan. Electronic address: [email protected].
- University of Chinese Academy of Sciences, Beijing, 101408, China. Electronic address: [email protected].
- Guangzhou Institutes of Biomedicine and Health, Chinese Academy of Sciences, Guangzhou, China. Electronic address: [email protected].
- Medical School, Medical College, Tianjin University, Tianjin, 300072, China. Electronic address: [email protected].
- Waseda University, Tokyo, 169-8050, Japan. Electronic address: [email protected].
- Hosei University, Chiyoda-ku, 102-8160, Japan. Electronic address: [email protected].
Abstract
While Large Language Models (LLMs) have revolutionized image captioning, their direct application to radiology report generation (RRG) faces two key challenges: the semantic dilution of lesion features due to indiscriminate global image encoding, and the stylistic drift resulting from LLMs' lack of strict clinical constraints. In this paper, we propose a Lesion-Aware Visio-Linguistic Context (LAVIC) regularization architecture, which synergizes focal anatomical attention with report structural consistency. In the visual encoding phase, we first design an anatomy-prioritized feature amplification pipeline that leverages a chest mask as explicit visual context to amplify the signal-to-noise ratio of chest-specific regions, thereby guiding the encoder to prioritize lesion-relevant semantics over background artifacts. Subsequently, we devise an anatomy-prioritized feature enhancement module to refine the expressiveness of the extracted features by modeling the implicit contextual dependencies among them. In the linguistic decoding phase, we mitigate the domain gap using regularization through complementary linguistic contexts. By injecting retrieved entity-level terminologies and summary-level report templates, we establish structural guardrails that regularize the LLM to adhere to radiological conventions without the need for retraining. LAVIC demonstrates strong NLG performance across three benchmarks and consistent advantages on clinically oriented automatic metrics for MIMIC-CXR and CheXPlus. Code will be released.