Segment any tumour: an uncertainty-aware vision foundation model for whole-body analysis.
Authors
Affiliations (7)
Affiliations (7)
- Department of Data Science & Artificial Intelligence, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia.
- Monash Biomedical Imaging (MBI), Monash University, Melbourne, VIC, Australia.
- Department of Electrical & Computer Systems Engineering, Faculty of Engineering, Monash University, Melbourne, VIC, Australia.
- Department of Neuroscience, The School of Translational Medicine, Monash University, Melbourne, VIC, Australia.
- Department of Radiology, Alfred Health, Melbourne, VIC, Australia.
- Department of Data Science & Artificial Intelligence, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia. [email protected].
- Monash Biomedical Imaging (MBI), Monash University, Melbourne, VIC, Australia. [email protected].
Abstract
Prompt-driven vision foundation models, such as the Segment Anything Model, have shown adaptability in computer vision, but their use in medical imaging remains challenging due to heterogeneous anatomy, artefacts, and low-contrast tumour boundaries. This is particularly difficult in whole-body tumour analysis, where models must transfer across modalities, anatomies, and tumour appearances. Here, we present Segment Any Tumour 3D (SAT3D), a lightweight volumetric foundation model for generalisable tumour segmentation across diverse medical imaging modalities, organs, and cohorts. SAT3D integrates a shifted-window vision transformer with critic-guided uncertainty-aware training, using confidence maps as dense prompts to guide boundary prediction in ambiguous regions. We benchmark SAT3D against vision foundation models, prompt-driven and task-specific methods across 11 public datasets. Trained on 17,075 three-dimensional volume-mask pairs, SAT3D shows robust generalisation, including in out-of-distribution settings, and is supported by a 3D-Slicer plugin for interactive segmentation, underscoring SAT3D's potential as a scalable foundation model for medical image analysis.