3D UXFormer: Dual-Branch Feature Fusion for Precise Breast MRI Multi-Tissue Segmentation.
Authors
Affiliations (15)
Affiliations (15)
- Institution of Computational Science and Technology, Guangzhou University, Guangzhou, 510006, China.
- Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangzhou, 510080, China.
- Department of Radiology, The First Hospital of China Medical University, Shenyang, 110001, China.
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China.
- Department of Radiology, Shanxi Province Cancer Hospital, Shanxi Hospital Affiliated to Cancer Hospital, Chinese Academy of Medical Sciences, Cancer Hospital Affiliated to Shanxi Medical University, Taiyuan, 030013, China.
- Department of Radiology, The Affiliated Cancer Hospital of Zhengzhou University & Henan Cancer Hospital, Zhengzhou, 450008, China.
- Department of Radiology, Affiliated Zhongshan Hospital of Dalian University, Dalian, 116001, China.
- Department of Radiology, The Fourth Affiliated Hospital of China Medical University, Shenyang, 110165, China.
- Medical Research Institute, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern MedicalUniversity, Guangzhou, 510080, China.
- Department of Medical Ultrasonics, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, 510120, China. [email protected].
- Institution of Computational Science and Technology, Guangzhou University, Guangzhou, 510006, China. [email protected].
- Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, China. [email protected].
- Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangzhou, 510080, China. [email protected].
- Medical Research Institute, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern MedicalUniversity, Guangzhou, 510080, China. [email protected].
- Institution of Computational Science and Technology, Guangzhou University, Guangzhou, 510006, China. [email protected].
Abstract
Accurate segmentation of breast tissues in MRI is essential for precise breast cancer diagnosis and effective treatment planning. Manually segmenting these tissues is not only arduous but also time-consuming. Fusing multi-phase MRI images, like the Contrast-enhanced Early ([Formula: see text]) and Contrast-enhanced Peak ([Formula: see text]) phases, to fully utilize their complementary information remains a challenge. In this paper, we introduce a novel 3D UXFormer model. It combines the 3D UNET and Vision Transformer (ViT) through a dual-branch structure, capitalizing on the 3D UNET's local feature extraction prowess and ViT's global feature-capturing ability. To address data-related issues, we pre-train ViT using the Masked Autoencoder (MAE) approach. Extensive experiments demonstrate that the 3D UXFormer outperforms state-of-the-art algorithms, achieving higher Dice and IoU scores across diverse breast tissues in different datasets. The source code of our method is available on Github.