Convolutional, State-Space and Transformer Liver-Tumor Segmentation Models Trained on Hepatocellular Carcinoma Fail on the Same External Cases: Lesion Size Outweighs Architecture in Cross-Dataset Generalization.
Authors
Affiliations (3)
Affiliations (3)
- Department of Biostatistics and Medical Informatics, Faculty of Medicine, Adıyaman University, 02040 Adıyaman, Türkiye.
- Department of Biostatistics and Medical Informatics, Faculty of Medicine, İnönü University, 44280 Malatya, Türkiye.
- Department of Surgery and Liver Transplantation, Faculty of Medicine, İnönü University, 44280 Malatya, Türkiye.
Abstract
Architectural innovation is widely assumed to improve the cross-institutional generalization of tumor segmentation models, yet this is rarely tested. We compared three architectural families under an identical, leak-safe protocol in hepatocellular carcinoma (HCC) segmentation. Convolutional (nnU-Net ResEnc-M), state-space (U-Mamba Bot) and transformer (SwinUNETR) models were trained on HCC-TACE-Seg (<i>n</i> = 103) with identical patient-disjoint five-fold splits, then applied without adaptation to 3D-IRCADb (<i>n</i> = 15) and LiTS (<i>n</i> = 118). We assessed discrimination, case-level failure concordance, lesion-volume dependence, calibration and selective prediction. Internal performance differed (Dice 0.685, 0.662, 0.479; Friedman <i>p</i> = 0.022) but external performance was low and largely convergent: the convolutional and state-space models were formally equivalent on both external cohorts (pre-specified margin ±0.05 Dice), whereas the transformer was inferior on LiTS (0.222; <i>p</i> < 0.0001) and inconclusive on 3D-IRCADb. Case-level Dice was highly concordant across all pairings (Spearman ρ = 0.84-0.96; 0.67-0.81 on LiTS after adjustment for lesion volume), the same cases failing across architectures. Lesion volume predicted performance in all six architecture × cohort combinations; lesions < 5 mL failed completely in 61-76% of cases and formed 1.9% of the development cohort against 31.4% of LiTS. Neither liver-mask constraint nor ensembling recovered performance. Deferring the 30% least-confident LiTS cases raised Dice to 0.516 (0.450 when only empty predictions were removed). Under a single-source, TACE-treated HCC training regime, cross-dataset failure of liver-tumor segmentation was governed primarily by lesion size rather than by architectural family; confidence-based selective prediction offers a deployable mitigation.