Swin-Qwen3: a three-stage vision-language framework for automated radiology report generation with multi-agent verification.

Authors

Abstract

Automated chest x-ray reporting could substantially reduce the burden on radiology services worldwide; however, the implementation of current vision-language models (VLMs) in clinical workflows remains limited by factual errors, hallucinations, and inadequate clinical reliability. Bridging this implementation gap requires frameworks that are not only technically sound but also designed for safe integration into real-world healthcare settings. This study aimed to improve the clinical accuracy and factual consistency of radiology report generation by introducing an agent-inspired, three-stage reasoning framework and evaluating its feasibility as research prototype for potential implementation in resource-constrained and high-throughput clinical environments. We propose the Swin-Qwen3 vision-language architecture, which integrates a Swin Transformer visual encoder with a Q-Former-style query-driven cross-attention module aligned with a large language model (Qwen3-0.6B). A three-stage generation strategy-comprising initial report drafting, clinical verification, and structured refinement-was guided by role-specific prompts to simulate distinct clinical reasoning behaviours. Parameter-efficient fine-tuning via low-rank adaptation (LoRA) enabled training within standard GPU constraints. The framework was evaluated on the full IU x-Ray test set (321 Samples) using lexical, semantic, clinical, and factuality metrics and compared with one- and two-stage ablations. Computational feasibility and inference overhead were assessed for offline or batch processing contexts. The three-stage framework showed modest but consistent improvements over ablation baselines. CheXpert-F1 reached 0.7038 (compared to 0.7154 for one-stage and 0.7123 for two-stage), Clinical-F1 reached 0.5401 (compared to 0.5478 for one-stage and 0.5449 for two-stage), and RadGraph-F1 scored 0.5467 (compared to 0.5478 for one-stage and 0.5445 for two-stage). The factuality score reached 0.843 (compared to 0.845 for one-stage and 0.845 for two-stage), while the hallucination rate remained at 0.6116 (compared to 0.6109 for one-stage and 0.6109 for two-stage). Clinical recall decreased from 0.8361 (one-stage) and 0.8337 (two-stage) to 0.8044 (three-stage), reflecting a trade-off between sensitivity and precision. However, the clinical verifier demonstrated substantial effectiveness, correcting 85.6% of identified hallucinations. Lexical quality improved: BLEU-4 was 0.0628 (+10.9% vs. one-stage; +13.1% vs. two-stage), ROUGE-2-F was 0.0957 (+10.2% vs. one-stage; +13.1% vs. two-stage), and METEOR was 0.3728 (+4.5% vs. one-stage; +6.3% vs. two-stage). Improvements in METEOR (<i>p</i> < 0.001) and ROUGE-2-F (<i>p</i> < 0.05) were statistically significant. A 14.8 ×  inference overhead was identified, indicating suitability for offline reporting contexts. The Swin-Qwen3 three-stage framework demonstrates that explicit clinical verification and iterative refinement can modestly enhance the clinical reliability of VLMs for automated radiology reporting. The parameter-efficient design is scalable across diverse clinical settings, including resource-limited environments, and supports a practical deployment pathway aligned with SDGs 3 (Good Health and Well-being), 9 (Industry, Innovation and Infrastructure), and 17 (Partnerships for the Goals). However, the residual hallucination rate of 61.2% indicates that the current system is a promising research prototype rather than a clinically deployable tool. Future work should prioritize multi-institutional validation on larger datasets (e.g., MIMIC-CXR) with radiologist expert review, regulatory evaluation, and prospective clinical integration studies.

Tags

Topics