MSMFusion: A Method for Multiscale Dual-Path State Space Modeling for Multimodal Medical Image Fusion.
Authors
Affiliations (2)
Affiliations (2)
- College of Computer Science and Technology, Changchun University, No. 6543, Satellite Road, Changchun 130022, China.
- Bethune First Clinical Medical College, Jilin University, Changchun 130021, China.
Abstract
Multimodal medical image fusion aims to integrate information from different imaging techniques to generate a single fused image that is comprehensive and rich in detail. Structural and functional imaging provide high-resolution anatomical details and tissue metabolic information, respectively. Complementary fusion improves quantitative fusion quality and preserves structural and functional information in one view, which aids clinical interpretation. Currently, convolutional neural network-based methods struggle to efficiently capture global features, whereas Transformer-based models excel at global modeling but incur relatively high computational costs. To address the above issues, we propose MSMFusion, a multiscale state-space modeling method for complementary structural-functional medical image fusion. A multiscale feature extraction module first extracts modality-specific multiscale structural features; subsequently, a dual-path MSMamba establishes long-range dependencies with linear complexity: the full-resolution scanning path preserves fine boundaries and textures, while the downsampled scanning path models global context with a more compact sequence; finally, a gated adaptive fusion module estimates spatial-channel weights based on local content to selectively integrate anatomical structure and functional metabolic information. On the Harvard medical image dataset, MSMFusion achieves a mutual information of 2.2181, a peak signal-to-noise ratio of 32.9390, and a structural similarity of 0.7616 on the MRI-PET task, and also outperforms various representative methods based on CNN, GAN, Transformer, and Mamba on the MRI-CT and MRI-SPECT tasks. Compared with the Transformer backbone, MSMamba reduces the number of parameters from 7.201M to 4.154M and the FLOPs from 476.738G to 159.239G, achieving superior fusion quality at lower computational cost while simultaneously preserving fine anatomical structures and key functional information.