BMR-Restormer: A transformer-based blind inpainting network for ultrasound marker removal.
Authors
Affiliations (10)
Affiliations (10)
- Shanghai Skin Disease Hospital, School of Medicine & Frontier Science Center for Stem Cell Research, Bioinformatics Department, School of Life Sciences and Technology, Tongji University, Shanghai, 200092, China; College of Information Engineering, China Jiliang University, Hangzhou, Zhejiang, 310018, China; Taizhou Campus, Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China; Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China.
- College of Information Engineering, China Jiliang University, Hangzhou, Zhejiang, 310018, China.
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China; Center of Intelligent Diagnosis and Therapy (Taizhou), Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Taizhou, Zhejiang, 317502, China.
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, Zhejiang, 310022, China.
- Department of Microbiology and Immunology, Graduate School of Medicine, Hokkaido University, Hokkaido, 001-0037, Japan.
- No.1 Hospital, Affiliated Qujing Hospital of Kunming Medical University, Qujing, Yunnan, 655000, China.
- Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, Zhejiang, 310022, China. Electronic address: [email protected].
- No.1 Hospital, Affiliated Qujing Hospital of Kunming Medical University, Qujing, Yunnan, 655000, China. Electronic address: [email protected].
- Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China; Department of Diagnostic Ultrasound Imaging & Interventional Therapy, Zhejiang Cancer Hospital, Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou, Zhejiang, 310022, China. Electronic address: [email protected].
- Shanghai Skin Disease Hospital, School of Medicine & Frontier Science Center for Stem Cell Research, Bioinformatics Department, School of Life Sciences and Technology, Tongji University, Shanghai, 200092, China. Electronic address: [email protected].
Abstract
Artificial markers, including crosses, text, and measurement lines, are commonly embedded in clinical ultrasound images to aid visual diagnosis. However, these overlays introduce substantial bias in deep learning models by drawing attention away from pathological structures, leading to degraded performance in computer-aided diagnosis (CAD). To address this issue, we propose the BMR-Restormer network specifically optimized for ultrasound marker removal. The architecture integrates Multi-Group Transposed Attention (MGTA) and Masked Window Self-Attention (MWSA) to enhance local-global feature modeling under low-contrast and speckle-rich conditions. A progressive learning strategy was employed to support coarse-to-fine feature acquisition. Our method achieved state-of-the-art image restoration performance, with PSNR/SSIM/LPIPS values of 63.68/0.999/0.003 on the internal dataset and 59.30/0.999/0.005 on the external dataset. To evaluate the clinical impact of artificial markers, we conducted three downstream tasks (classification, segmentation, and detection) under different training-testing configurations. The results showed that models trained on marker-containing data had markedly lower generalization performance when applied to marker-free images. For example, classification precision dropped from 0.8830 to 0.5484 when the model trained on marker-containing data was evaluated on clean images. Similarly, segmentation Dice scores fell from 0.8217 to 0.4759. In object detection, AP@50 decreased from 0.9928 to 0.2393. These results demonstrated that artificial markers cause models to overfit spurious features and shift attention away from diagnostically relevant regions. In contrast, models trained on marker-free data maintained high accuracy on both clean and marked images, confirming that the proposed BMR-Restormer provides a robust and clinically reliable solution for ultrasound marker removal.